-- Logs begin at Mon 2022-07-18 14:14:42 UTC, end at Mon 2022-07-18 15:32:49 UTC. -- Jul 18 14:51:27.686747 dv-cinder-836973-13-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 18 14:51:27.696134 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 18 14:51:27.698243 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 18 14:51:27 2022] *** Jul 18 14:51:27.698243 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 18 14:51:27.698243 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 18 14:51:27.698243 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: nodename: dv-cinder-836973-13-iscsi Jul 18 14:51:27.698243 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: machine: x86_64 Jul 18 14:51:27.698243 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: clock source: unix Jul 18 14:51:27.698243 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: pcre jit disabled Jul 18 14:51:27.698243 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: detected number of CPU cores: 4 Jul 18 14:51:27.698243 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: current working directory: / Jul 18 14:51:27.698243 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: detected binary path: /usr/bin/uwsgi-core Jul 18 14:51:27.698243 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: your processes number limit is 47680 Jul 18 14:51:27.699007 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: your memory page size is 4096 bytes Jul 18 14:51:27.699007 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: detected max file descriptor number: 2048 Jul 18 14:51:27.699007 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: lock engine: pthread robust mutexes Jul 18 14:51:27.699007 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: thunder lock: enabled Jul 18 14:51:27.699007 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 18 14:51:27.699007 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 18 14:51:27.741837 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: Python main interpreter initialized at 0x556faa9829c0 Jul 18 14:51:27.741837 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: python threads support enabled Jul 18 14:51:27.741837 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: your server socket listen backlog is limited to 100 connections Jul 18 14:51:27.741837 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: your mercy for graceful operations on workers is 90 seconds Jul 18 14:51:27.742232 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: mapped 403077 bytes (393 KB) for 2 cores Jul 18 14:51:27.742318 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: *** Operational MODE: preforking *** Jul 18 14:51:27.742586 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: *** uWSGI is running in multiple interpreter mode *** Jul 18 14:51:27.742586 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: spawned uWSGI master process (pid: 80292) Jul 18 14:51:27.742817 dv-cinder-836973-13-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 18 14:51:27.744173 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: spawned uWSGI worker 1 (pid: 80303, cores: 1) Jul 18 14:51:27.744173 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: spawned uWSGI worker 2 (pid: 80304, cores: 1) Jul 18 14:51:27.744173 dv-cinder-836973-13-iscsi devstack@keystone.service[80292]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 18 14:51:33.339502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 14:51:33.339502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: from cryptography.utils import int_from_bytes Jul 18 14:51:33.340370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 14:51:33.340370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: from cryptography.utils import int_from_bytes Jul 18 14:51:33.340488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 14:51:33.340488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: from cryptography.utils import int_from_bytes Jul 18 14:51:33.340925 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 14:51:33.340925 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: from cryptography.utils import int_from_bytes Jul 18 14:51:33.390271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 14:51:33.390507 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 14:51:33.390677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] command line args: [] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 14:51:33.390812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 14:51:33.390961 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 14:51:33.391058 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 14:51:33.391145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ================================================================================ {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 14:51:33.391227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] command line args: [] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 14:51:33.391378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] admin_token = **** {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.391560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 14:51:33.391705 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] config_dir = [] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.391882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ================================================================================ {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 14:51:33.391997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.392083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] config_source = [] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.392210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] admin_token = **** {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.392301 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] debug = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.392468 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.392711 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] config_dir = [] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.392711 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.392841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.392903 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] config_source = [] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.392961 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.393018 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] debug = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.393245 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.393355 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.393427 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.393494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.393658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.393756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] list_limit = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.393826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] log_config_append = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.393892 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.394050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.394134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.394208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] log_dir = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.394372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] list_limit = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.394464 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] log_file = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.394526 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] log_config_append = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.394633 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.394696 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.394845 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.394937 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] log_dir = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.395100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] log_file = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.395238 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.395330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.395413 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.395503 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.395585 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.395665 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.395782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.395892 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.396038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.396150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.396234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.396341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.396455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.396576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.396690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.396811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.396917 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.397046 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.397154 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.397295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.397397 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.397540 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.397646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.397785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.397890 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.398039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.398291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.83/identity {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.398382 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.398458 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] publish_errors = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.398596 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.398728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.398963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.399087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.83/identity {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.399175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.399278 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] publish_errors = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.399380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.399518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.399606 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.399758 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.399911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] standard_threads = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.400002 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.400121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.400240 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.400323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.400459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.400544 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.400618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] use_journal = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.400689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] standard_threads = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.400818 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] use_json = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.400902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.401060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] use_stderr = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.401138 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.401216 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] use_syslog = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.401289 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.401366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.401438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] use_journal = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.401569 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.401652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] use_json = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.401778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] use_stderr = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.401858 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.401962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] use_syslog = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.402051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.402213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:51:33.402331 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.402465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.402612 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.402750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.402840 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.403012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.403098 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.403248 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.403336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.403499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.403618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.403744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.403824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.403994 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.404079 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.404231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.404313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.404483 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.404566 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.404720 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.404801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.404967 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.405048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.405208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.405294 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.405446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.405531 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.405697 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.405776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.405952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.406040 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.406202 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.406318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.406441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.406591 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.406756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.406839 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.406976 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.407593 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.407845 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.407932 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.408100 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.408186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.408455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.408536 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.408708 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.408796 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.408946 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.409032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.409185 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.409320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.409426 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.409629 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.409731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.409875 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.409969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.410163 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.410254 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.410412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.410511 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.410700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.410822 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.411649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.411773 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.411913 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.411994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.412112 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.412237 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.412317 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.412476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.412572 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.412648 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.412779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.412864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.412990 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.413113 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.413203 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.413329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.413413 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.413494 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.413610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.413747 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.413841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.413926 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.414006 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.414133 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.414219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.414295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.414418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.414505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.414630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.414760 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.414851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.414927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.415058 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.415190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.415271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.415402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.415532 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.415618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.415742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.415830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.415830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.416025 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.416103 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.416176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.416300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.416394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.416521 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.416606 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.416751 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.416895 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.416999 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.417161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.417245 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.417433 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.417521 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.417704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.417785 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.417964 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.418059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.418209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.418292 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.418456 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.418594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.418722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.418807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.418991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.419075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.419236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.419348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.419502 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.419622 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.419706 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.419909 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.419993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.420189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.420284 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.420432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.420512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.420688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.420771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.420941 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.421032 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.421186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.421278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.421426 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.421556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.421667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.421827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.421917 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.422110 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.422194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.422382 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.422464 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.422669 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.423418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.423522 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.423691 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.423772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.423935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.424045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.424179 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.424320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.424411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.424624 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.424713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.424901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.424990 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.425148 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.425231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.425393 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.425474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.425644 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.425725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.425889 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.425971 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.426134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.426216 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.426390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.426472 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.426663 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.426816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.426898 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.427085 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.427178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.427427 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.427570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.427657 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.427730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.427888 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.427968 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.428133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.428223 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.428373 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.428456 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.428624 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.428706 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.428865 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.428948 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.429125 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.429219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.429389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.429473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.429631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] auth.external = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.429728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.429864 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] auth.external = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.429951 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.430098 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.430186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.430396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.430487 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.430646 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.430726 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] auth.password = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.430887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] auth.password = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.430971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] auth.token = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.431123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] auth.token = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.431213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.431365 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.431454 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.431603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.431683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.431846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.431927 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.432078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.432160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.432327 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.432409 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.432574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.432657 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.432806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.432888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] credential.caching = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.433043 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] credential.caching = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.433126 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.433292 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.433375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.433543 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.433636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.433788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.433869 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.434024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.434105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.434270 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.434354 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.434505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.434607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.434786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.434872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.435028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.435114 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.435270 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.435353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.435525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.435607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.435811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.435895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.436058 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.436140 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.436316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.436397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.436559 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.436641 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.436816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] federation.caching = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.436897 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.437052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] federation.caching = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.437132 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.437293 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.437377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.437542 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.437634 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.437778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.437861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.438011 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.438093 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.438259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.438361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.438609 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.438755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.438854 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.438933 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.439045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.439287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.439369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.439531 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.439612 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.439782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.439865 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.440020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity.caching = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.440102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity.caching = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.440270 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.440354 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.440524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.440647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.440761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.440862 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.440997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.441089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.441247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.441330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.441480 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.441562 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.441720 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.441804 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.441972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.442063 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.442226 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.442310 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.442458 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.442565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.442716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.442801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.442950 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.443032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.443196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.443282 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.443447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.443538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.443701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.443783 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.443958 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.444042 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.444209 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.444291 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.444483 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.444565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.444724 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.444808 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.444964 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.445053 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.445204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.445286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.445444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.445528 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.445677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.445759 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.445913 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.445996 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.446279 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.446363 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.446528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.446641 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.446807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.446889 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.447046 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.447131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.447300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.447384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.447552 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.447635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.447793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.447921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.448043 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.448125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.448293 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.448377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.448529 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.448613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.448771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.448853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.449006 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.449088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.449248 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.449333 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.449489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.449572 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.449735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.449817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.449975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.450058 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.450214 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.450297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.450470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.450573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.450747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.450833 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.450984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.451068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.451219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.451311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.451471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.451553 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.451722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.451805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.451960 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.452051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.452197 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.452280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.452434 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.452517 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.452673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.452796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.452911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.452993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.453173 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.453255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.453411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.453500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.453657 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.453738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.453923 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.454006 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.454175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.454412 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.454661 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.454799 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.454938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.455075 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.455208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.455355 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.455490 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.455623 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.455776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.455910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.456045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.456186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.456326 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.456460 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.456595 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.456731 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.456866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.457007 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.457141 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.457274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.457404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.457539 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.457685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.457829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.457964 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.458104 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.458237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.458324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.458387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.458457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.458521 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.458722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.458841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.458981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.459114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.459114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.459443 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.459582 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.459721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.459853 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.459984 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.460124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.460255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.460393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.460524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.460654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.460785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.460924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.461055 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.461195 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.461326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.461456 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.461587 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.461726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.461858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.461990 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.462120 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.462251 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] resource.caching = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.462381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.462521 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.462678 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.462758 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.462832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.462905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.462978 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] resource.caching = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.463069 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.463148 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.463222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.463296 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.463369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.463441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.463523 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.463599 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.463672 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.463745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.463817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.463889 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.463970 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] role.caching = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.464047 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.464119 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] role.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.464192 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.464264 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.464337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.464419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.464495 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] role.caching = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.464567 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.464731 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] role.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.464814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.464971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.465055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.465230 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.465313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.465486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.465569 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.465743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.465826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.465994 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.466076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.466229 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.466311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.466480 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.466637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.466757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.466840 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.467008 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.467091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.467253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.467392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.467529 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.467693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.467832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.467966 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.468108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.468244 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.468380 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.468519 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.468673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.468817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.468964 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.469098 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.469231 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.469367 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.469500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.470115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.470266 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.470404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.470560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.470693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.470829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.470975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.470975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.471239 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.471380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.471515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.471650 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.471796 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.471930 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.472063 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.472198 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.472332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.472467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.472608 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.472744 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.472886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.473033 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.473164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.473299 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] token.caching = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.473442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.473579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.473711 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.473846 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.473980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.474114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.474254 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.474389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.474524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] token.caching = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.474673 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.474814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.474946 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.475088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.475221 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.475353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.475484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.475615 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.475746 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.475885 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.476017 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.476148 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.476278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.476409 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.476540 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.476688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.476825 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.476957 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.477089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.477219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.477350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.477489 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.477621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.477753 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.477884 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.478015 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.478147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.connection = **** {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.478290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.478443 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.478624 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.478760 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.478895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.479030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.479163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.479308 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.479308 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.479559 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.connection = **** {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.479701 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.479911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.480048 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.480185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.480314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.480468 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.480602 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.480737 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.480872 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.481003 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.481137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.481279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.481412 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.481551 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.481684 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.481818 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.481952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.482093 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.482228 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.482390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.482529 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.482682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.482819 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.482962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.483099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.483235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.483366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80303) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 14:51:33.483502 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.483502 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.483502 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.483502 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.483502 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:51:33.483502 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80304) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 14:51:33.513147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.513422 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.513564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.513698 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.513833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.513968 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.522506 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.522902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.550581 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.550853 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.551303 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.551546 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.696190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.697662 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.745942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.746184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.746777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.747090 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.811866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.812082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.812237 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.812335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.812505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.812646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.812737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.812926 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.813014 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.813370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.813692 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.813987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.839059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.839283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.839489 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.839607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.839720 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.839904 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.839993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.840146 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.840304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.840394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.840652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.840958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.841250 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.841552 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.852295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.852965 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.865496 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.865729 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.865821 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.865956 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.866177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.866264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.866400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.866633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.866738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.866850 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80303) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.867035 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.867332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.867614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.867901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80304) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:51:33.884193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.884343 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.884422 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 14:51:33.884729 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.884814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 14:51:33.884938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 14:51:33.885231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.885320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.885497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.885583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 14:51:33.885757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.885974 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.886060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.886355 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.886723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.887059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.897683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.897959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.898231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.898512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.899310 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.899662 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.900006 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.900363 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.906693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:33.909418 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:33.910796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:33.911079 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.911362 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.911753 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.912038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.912301 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.912575 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.912850 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.913119 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.913572 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.913675 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:33.913765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.913844 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.914168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.914678 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.915038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.915376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.915723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.916072 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.916419 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.916780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.917123 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.940518 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.942156 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:33.942435 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.944641 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.946417 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:33.946797 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.947750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:33.948029 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.948314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.952320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:33.952609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.952902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.956241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:33.956524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.960908 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:33.961165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:33.961258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.961451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.961734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.962019 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.962296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.962590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.962864 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.963134 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.965872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:33.966167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.966457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.966776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.967061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.967344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.967624 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.967902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.984102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.984389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.988180 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.988477 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.989390 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:33.993120 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.993213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:33.993398 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.997005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.997310 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:33.998406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.002385 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.002491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.006065 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.006165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.006434 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.006749 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.007036 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.007331 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.007627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.009897 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.010206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.010498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.010819 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.011123 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.011412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.027307 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.027686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.027957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.028230 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.031124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.031515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.031795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.032657 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.037006 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.038758 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.039034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.040819 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.042072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.042363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.042628 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.042771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.042909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.043045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.043240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.045979 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.046270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.046580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.046882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.047173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.057715 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.057996 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.058278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.058584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.058867 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.059157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.059435 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.061293 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.061578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.061887 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.062154 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.062439 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.062762 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.063048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.079377 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.082674 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.083964 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.085983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.087317 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.087604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.089253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.089531 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.089815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.090683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.092338 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.094000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.094289 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.095109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.097925 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.099671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.102678 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.103063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.103355 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.103640 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.103916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.104295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.107656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.107954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.108247 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.108532 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.113632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.113919 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.114200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.118188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.118477 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.118792 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.122033 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80303) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.122316 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.122630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.122909 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.123192 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.123482 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.123756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.124025 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.124296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.124574 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.124858 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.125130 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.125395 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.125661 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80303) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.127503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80304) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:51:34.127792 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.128087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.128372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.128653 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.128954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.129237 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.129523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.129801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.130097 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.130379 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.130681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.130957 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.131232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80304) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:51:34.170154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x556faa9829c0 pid: 80303 (default app) Jul 18 14:51:34.170451 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x556faa9829c0 pid: 80304 (default app) Jul 18 14:51:34.173702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b1118-5d33-4737-bef2-01a9e3e4a86c None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:34.173993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b1118-5d33-4737-bef2-01a9e3e4a86c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:34.174272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b1118-5d33-4737-bef2-01a9e3e4a86c None None] PATH_INFO: `/v3` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:34.178580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1/1] 10.222.0.83 () {50 vars in 791 bytes} [Mon Jul 18 14:51:34 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:51:42.975119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb1aec-bfdc-46da-b670-d0f330dd5a3d None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:42.975649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb1aec-bfdc-46da-b670-d0f330dd5a3d None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:42.975649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb1aec-bfdc-46da-b670-d0f330dd5a3d None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:42.980036 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1/2] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:51:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:51:42.986755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732fa9f5-ccc4-4705-b467-673134b4d4f8 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:42.987256 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732fa9f5-ccc4-4705-b467-673134b4d4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:42.987381 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732fa9f5-ccc4-4705-b467-673134b4d4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:43.057822 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: INFO dbcounter [None req-732fa9f5-ccc4-4705-b467-673134b4d4f8 None None] Registered counter for database keystone Jul 18 14:51:43.080472 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG oslo_db.sqlalchemy.engines [None req-732fa9f5-ccc4-4705-b467-673134b4d4f8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 14:51:43.087094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writer thread running {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 14:51:43.271161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG passlib.handlers.bcrypt [None req-732fa9f5-ccc4-4705-b467-673134b4d4f8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80303) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 14:51:43.271577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG passlib.handlers.bcrypt [None req-732fa9f5-ccc4-4705-b467-673134b4d4f8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80303) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 14:51:43.327820 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-732fa9f5-ccc4-4705-b467-673134b4d4f8 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:51:43.329554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-732fa9f5-ccc4-4705-b467-673134b4d4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:51:43.495948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 2/3] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:51:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 511 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 18 14:51:43.641168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb1aec-bfdc-46da-b670-d0f330dd5a3d None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:43.712439 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: INFO dbcounter [None req-4aeb1aec-bfdc-46da-b670-d0f330dd5a3d None None] Registered counter for database keystone Jul 18 14:51:43.733123 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG oslo_db.sqlalchemy.engines [None req-4aeb1aec-bfdc-46da-b670-d0f330dd5a3d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80304) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 14:51:43.741063 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writer thread running {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 14:51:43.863830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb1aec-bfdc-46da-b670-d0f330dd5a3d None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:43.868161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52863c10-f61f-49b9-bd6b-0acf61c37b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:43.868829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52863c10-f61f-49b9-bd6b-0acf61c37b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:43.869122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52863c10-f61f-49b9-bd6b-0acf61c37b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:43.869405 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52863c10-f61f-49b9-bd6b-0acf61c37b08 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:43.879266 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52863c10-f61f-49b9-bd6b-0acf61c37b08 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:43.934381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: WARNING keystone.common.rbac_enforcer.enforcer [None req-52863c10-f61f-49b9-bd6b-0acf61c37b08 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 14:51:43.934777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52863c10-f61f-49b9-bd6b-0acf61c37b08 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:43.945885 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 2/4] 10.222.0.83 () {58 vars in 1324 bytes} [Mon Jul 18 14:51:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:51:43.951976 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732fa9f5-ccc4-4705-b467-673134b4d4f8 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:43.967678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732fa9f5-ccc4-4705-b467-673134b4d4f8 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:43.972323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff59649-7e46-4a2f-a1a6-bee946679714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:43.973468 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff59649-7e46-4a2f-a1a6-bee946679714 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:43.973760 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff59649-7e46-4a2f-a1a6-bee946679714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:43.974041 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff59649-7e46-4a2f-a1a6-bee946679714 admin admin] PATH_INFO: `/v3/projects/8015bf50f271424ca62c239a2df319f8` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:43.978215 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff59649-7e46-4a2f-a1a6-bee946679714 admin admin] RBAC: Authorizing `identity:get_project(project_id=8015bf50f271424ca62c239a2df319f8)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:44.030780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3ff59649-7e46-4a2f-a1a6-bee946679714 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 14:51:44.031446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff59649-7e46-4a2f-a1a6-bee946679714 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:44.036014 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 3/5] 10.222.0.83 () {56 vars in 1207 bytes} [Mon Jul 18 14:51:43 2022] GET /identity/v3/projects/8015bf50f271424ca62c239a2df319f8 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:51:46.085209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cab29e-0214-4b08-a682-a7bc67318cd7 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:46.085747 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cab29e-0214-4b08-a682-a7bc67318cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:46.085902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cab29e-0214-4b08-a682-a7bc67318cd7 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:46.091166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 3/6] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:51:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:51:46.098097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd1e1d-5cb5-4f79-b1c0-3ebad3426b75 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:46.098612 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd1e1d-5cb5-4f79-b1c0-3ebad3426b75 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:46.098703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd1e1d-5cb5-4f79-b1c0-3ebad3426b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:46.119060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-aefd1e1d-5cb5-4f79-b1c0-3ebad3426b75 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:51:46.120634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-aefd1e1d-5cb5-4f79-b1c0-3ebad3426b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:51:46.128893 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 4/7] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:51:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 14:51:46.151461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cab29e-0214-4b08-a682-a7bc67318cd7 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:46.165080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cab29e-0214-4b08-a682-a7bc67318cd7 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:46.169430 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb41c1-6dfd-4c15-8452-4fbb506260ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:46.170085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb41c1-6dfd-4c15-8452-4fbb506260ca admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:46.170379 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb41c1-6dfd-4c15-8452-4fbb506260ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:46.170707 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb41c1-6dfd-4c15-8452-4fbb506260ca admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:46.179486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb41c1-6dfd-4c15-8452-4fbb506260ca admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:46.181233 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb41c1-6dfd-4c15-8452-4fbb506260ca admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:46.190939 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 4/8] 10.222.0.83 () {58 vars in 1324 bytes} [Mon Jul 18 14:51:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:51:46.196952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefd1e1d-5cb5-4f79-b1c0-3ebad3426b75 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:46.206668 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefd1e1d-5cb5-4f79-b1c0-3ebad3426b75 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:46.211102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe0852-4592-4b21-b82d-5ce19bb77553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:46.211973 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe0852-4592-4b21-b82d-5ce19bb77553 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:46.212258 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe0852-4592-4b21-b82d-5ce19bb77553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:46.212539 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe0852-4592-4b21-b82d-5ce19bb77553 admin admin] PATH_INFO: `/v3/users/baf04811d48d4c3fa638546a709d7179` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:46.213936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe0852-4592-4b21-b82d-5ce19bb77553 admin admin] RBAC: Authorizing `identity:get_user(user_id=baf04811d48d4c3fa638546a709d7179)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:46.215822 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe0852-4592-4b21-b82d-5ce19bb77553 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:46.217434 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 5/9] 10.222.0.83 () {56 vars in 1198 bytes} [Mon Jul 18 14:51:46 2022] GET /identity/v3/users/baf04811d48d4c3fa638546a709d7179 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:51:48.346642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b52cdf-7104-4644-bc76-f68434b7bea0 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:48.347193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b52cdf-7104-4644-bc76-f68434b7bea0 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:48.347193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b52cdf-7104-4644-bc76-f68434b7bea0 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:48.348568 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 5/10] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:51:48.354003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d3197-3a90-4a6e-91dd-c120c60bd1b5 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:48.354441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d3197-3a90-4a6e-91dd-c120c60bd1b5 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:48.354715 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d3197-3a90-4a6e-91dd-c120c60bd1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:48.372314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-d83d3197-3a90-4a6e-91dd-c120c60bd1b5 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:51:48.373348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d83d3197-3a90-4a6e-91dd-c120c60bd1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:51:48.408977 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4decce-52ce-45d8-9f3a-d58301672505 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:48.409264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4decce-52ce-45d8-9f3a-d58301672505 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:48.409549 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4decce-52ce-45d8-9f3a-d58301672505 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:48.410948 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 6/11] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:51:48.416169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc846a60-e7a1-4b36-a810-6f52e39df797 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:48.416464 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc846a60-e7a1-4b36-a810-6f52e39df797 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:48.416748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc846a60-e7a1-4b36-a810-6f52e39df797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:48.457841 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG passlib.handlers.bcrypt [None req-dc846a60-e7a1-4b36-a810-6f52e39df797 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80304) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 14:51:48.458267 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG passlib.handlers.bcrypt [None req-dc846a60-e7a1-4b36-a810-6f52e39df797 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80304) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 14:51:48.459736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 6/12] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:51:48.483153 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d3197-3a90-4a6e-91dd-c120c60bd1b5 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:48.504383 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-dc846a60-e7a1-4b36-a810-6f52e39df797 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:51:48.506035 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-dc846a60-e7a1-4b36-a810-6f52e39df797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:51:48.507948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:48.594760 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d3197-3a90-4a6e-91dd-c120c60bd1b5 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:48.613142 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 7/13] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:51:48.629758 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424340d-a6d4-4952-9106-af06ce7b9c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:48.630606 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424340d-a6d4-4952-9106-af06ce7b9c7e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:48.630899 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424340d-a6d4-4952-9106-af06ce7b9c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:48.631182 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424340d-a6d4-4952-9106-af06ce7b9c7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:48.635864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc846a60-e7a1-4b36-a810-6f52e39df797 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:48.644093 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424340d-a6d4-4952-9106-af06ce7b9c7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:51:48.655671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-4424340d-a6d4-4952-9106-af06ce7b9c7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:51:48.655671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:51:48.655671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:51:48.655671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:51:48.655671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:51:48.655671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:51:48.655671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:51:48.655671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:51:48.655671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:51:48.655671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:51:48.655671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:51:48.655671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:51:48.655671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:51:48.655671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:51:48.655671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:51:48.655671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:51:48.656782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:51:48.656782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:51:48.656782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:51:48.656782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:51:48.656782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:51:48.656782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:51:48.656782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:51:48.656782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:51:48.656782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:51:48.656782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:51:48.656782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:51:48.656782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:51:48.656782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:51:48.656782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:51:48.656782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:51:48.656782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:51:48.658034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:51:48.658034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:51:48.658034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:51:48.658034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:51:48.658034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:51:48.658034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:51:48.658034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:51:48.658034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:51:48.658034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:51:48.658034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:51:48.658034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:51:48.658034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:51:48.658034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 7/14] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:51:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:51:48.663285 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424340d-a6d4-4952-9106-af06ce7b9c7e None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:48.741581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc846a60-e7a1-4b36-a810-6f52e39df797 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:48.744019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:48.752032 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:48.770517 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424340d-a6d4-4952-9106-af06ce7b9c7e None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:48.780120 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd932eed-8d7a-4b5d-80ae-4f700f59cb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:48.781038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd932eed-8d7a-4b5d-80ae-4f700f59cb3f None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:48.781346 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd932eed-8d7a-4b5d-80ae-4f700f59cb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:48.781636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd932eed-8d7a-4b5d-80ae-4f700f59cb3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:48.806980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf5ad7-73b8-4e3a-a701-c203715c6258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:48.807660 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf5ad7-73b8-4e3a-a701-c203715c6258 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:48.807961 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf5ad7-73b8-4e3a-a701-c203715c6258 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:48.808247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf5ad7-73b8-4e3a-a701-c203715c6258 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:48.809020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf5ad7-73b8-4e3a-a701-c203715c6258 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:51:48.809330 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf5ad7-73b8-4e3a-a701-c203715c6258 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:48.810792 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf5ad7-73b8-4e3a-a701-c203715c6258 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:48.823650 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 8/15] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:51:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:51:48.829411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf5ad7-73b8-4e3a-a701-c203715c6258 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:48.854195 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd932eed-8d7a-4b5d-80ae-4f700f59cb3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:48.855795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd932eed-8d7a-4b5d-80ae-4f700f59cb3f None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:48.936984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf5ad7-73b8-4e3a-a701-c203715c6258 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:48.962379 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 8/16] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:51:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 14:51:48.968092 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd932eed-8d7a-4b5d-80ae-4f700f59cb3f None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:48.971238 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:48.973504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27521b73-84c1-40f7-853f-3b58da096ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:48.974405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27521b73-84c1-40f7-853f-3b58da096ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:48.974724 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27521b73-84c1-40f7-853f-3b58da096ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:48.975001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27521b73-84c1-40f7-853f-3b58da096ce9 None admin] PATH_INFO: `/v3/users/baf04811d48d4c3fa638546a709d7179` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:48.976246 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27521b73-84c1-40f7-853f-3b58da096ce9 None admin] RBAC: Authorizing `identity:get_user(user_id=baf04811d48d4c3fa638546a709d7179)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:48.977743 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27521b73-84c1-40f7-853f-3b58da096ce9 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:48.979760 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 9/17] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:51:48 2022] GET /identity/v3/users/baf04811d48d4c3fa638546a709d7179 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:51:48.986471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27521b73-84c1-40f7-853f-3b58da096ce9 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:49.003770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:49.079239 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd932eed-8d7a-4b5d-80ae-4f700f59cb3f None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:49.094994 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27521b73-84c1-40f7-853f-3b58da096ce9 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:49.115769 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32362674-61bd-477b-bfd9-b6c67a290573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:49.116579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32362674-61bd-477b-bfd9-b6c67a290573 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:49.116862 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32362674-61bd-477b-bfd9-b6c67a290573 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:49.117143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32362674-61bd-477b-bfd9-b6c67a290573 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:49.133983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482c598-bd4b-48e8-9687-ee91d99d1554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:49.134824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482c598-bd4b-48e8-9687-ee91d99d1554 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:49.135180 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482c598-bd4b-48e8-9687-ee91d99d1554 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:49.135461 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482c598-bd4b-48e8-9687-ee91d99d1554 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:49.136676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482c598-bd4b-48e8-9687-ee91d99d1554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:49.137495 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32362674-61bd-477b-bfd9-b6c67a290573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:49.138172 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482c598-bd4b-48e8-9687-ee91d99d1554 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:49.139086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32362674-61bd-477b-bfd9-b6c67a290573 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:49.139858 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 10/18] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:51:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:51:49.146079 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482c598-bd4b-48e8-9687-ee91d99d1554 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:49.161164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-32362674-61bd-477b-bfd9-b6c67a290573 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:51:49.161164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:51:49.161164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:51:49.161164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:51:49.161164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:51:49.161164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:51:49.161164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:51:49.161164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:51:49.161164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:51:49.161164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:51:49.161164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:51:49.161164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:51:49.161164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:51:49.161164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:51:49.161164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:51:49.162101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:51:49.162952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:51:49.162952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:51:49.162952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:51:49.162952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:51:49.162952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 9/19] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:51:49.169502 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32362674-61bd-477b-bfd9-b6c67a290573 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:49.233663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:49.236076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:49.255805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482c598-bd4b-48e8-9687-ee91d99d1554 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:49.282774 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32362674-61bd-477b-bfd9-b6c67a290573 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:49.291627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4832c-a026-48fd-baf6-33c650e9770a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:49.292297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4832c-a026-48fd-baf6-33c650e9770a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:49.292591 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4832c-a026-48fd-baf6-33c650e9770a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:49.292947 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4832c-a026-48fd-baf6-33c650e9770a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:49.293908 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4832c-a026-48fd-baf6-33c650e9770a None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.domain.id=default, user.id=baf04811d48d4c3fa638546a709d7179) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:51:49.294341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4832c-a026-48fd-baf6-33c650e9770a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:49.295778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4832c-a026-48fd-baf6-33c650e9770a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:49.304601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 11/20] 10.222.0.83 () {56 vars in 1322 bytes} [Mon Jul 18 14:51:49 2022] GET /identity/v3/role_assignments?user.id=baf04811d48d4c3fa638546a709d7179&scope.domain.id=default&role.id=5891920310b54787bd1152d736d2a3f1 => generated 233 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:51:49.319452 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e1f69-908f-48f4-99f2-dffa69797ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:49.320151 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e1f69-908f-48f4-99f2-dffa69797ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:49.320438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e1f69-908f-48f4-99f2-dffa69797ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:49.320720 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e1f69-908f-48f4-99f2-dffa69797ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:49.321487 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e1f69-908f-48f4-99f2-dffa69797ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:51:49.321799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e1f69-908f-48f4-99f2-dffa69797ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:49.323315 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e1f69-908f-48f4-99f2-dffa69797ed4 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:49.344906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 10/21] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:51:51.674588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459847a1-eb32-47c0-b40a-3255d46d7ef9 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:51.675651 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459847a1-eb32-47c0-b40a-3255d46d7ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:51.676176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459847a1-eb32-47c0-b40a-3255d46d7ef9 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:51.677764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 12/22] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:51:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:51:51.683414 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb6de8-e7ca-43f1-b83c-015d113f8565 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:51.684333 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb6de8-e7ca-43f1-b83c-015d113f8565 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:51.684824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb6de8-e7ca-43f1-b83c-015d113f8565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:51.710843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-c9eb6de8-e7ca-43f1-b83c-015d113f8565 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:51:51.712444 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c9eb6de8-e7ca-43f1-b83c-015d113f8565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:51:51.815962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 11/23] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:51:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:51:51.866057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459847a1-eb32-47c0-b40a-3255d46d7ef9 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:52.009420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459847a1-eb32-47c0-b40a-3255d46d7ef9 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:52.054328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77c2a0-caaf-4c12-b194-cd0a4ab8a6ab None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:52.054892 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77c2a0-caaf-4c12-b194-cd0a4ab8a6ab None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:52.055353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77c2a0-caaf-4c12-b194-cd0a4ab8a6ab None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:52.059507 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 12/24] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:51:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:51:52.071577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544f68f-a7b5-4ab5-a852-8f527687d016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:52.072383 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544f68f-a7b5-4ab5-a852-8f527687d016 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:52.072673 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544f68f-a7b5-4ab5-a852-8f527687d016 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:52.072776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a6c10-f161-47b1-8ce3-7ec3f4b6b572 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:52.072953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544f68f-a7b5-4ab5-a852-8f527687d016 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:52.073433 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a6c10-f161-47b1-8ce3-7ec3f4b6b572 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:52.073925 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a6c10-f161-47b1-8ce3-7ec3f4b6b572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:52.085072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544f68f-a7b5-4ab5-a852-8f527687d016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:51:52.094980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-6544f68f-a7b5-4ab5-a852-8f527687d016 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:51:52.094980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:51:52.094980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:51:52.094980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:51:52.094980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:51:52.094980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:51:52.094980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:51:52.094980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:51:52.094980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:51:52.094980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:51:52.094980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:51:52.094980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:51:52.094980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:51:52.094980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:51:52.094980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:51:52.094980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:51:52.095968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:51:52.095968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:51:52.095968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:51:52.095968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:51:52.095968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:51:52.095968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:51:52.095968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:51:52.095968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:51:52.095968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:51:52.095968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:51:52.095968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:51:52.095968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:51:52.095968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:51:52.095968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:51:52.095968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:51:52.095968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:51:52.096717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:51:52.096717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:51:52.096717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:51:52.096717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:51:52.096717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:51:52.096717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:51:52.096717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:51:52.096717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:51:52.096717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:51:52.096717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:51:52.096717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:51:52.096717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:51:52.096717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 13/25] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:51:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:51:52.102455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544f68f-a7b5-4ab5-a852-8f527687d016 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:52.107238 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:52.107754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-1b3a6c10-f161-47b1-8ce3-7ec3f4b6b572 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:51:52.109005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-1b3a6c10-f161-47b1-8ce3-7ec3f4b6b572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:51:52.216847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:52.284314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544f68f-a7b5-4ab5-a852-8f527687d016 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:52.330437 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37a3b9-6423-4518-a0d6-30d6c7ef3edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:52.331419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37a3b9-6423-4518-a0d6-30d6c7ef3edc None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:52.331862 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37a3b9-6423-4518-a0d6-30d6c7ef3edc None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:52.331969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 13/26] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:51:52.332399 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37a3b9-6423-4518-a0d6-30d6c7ef3edc None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:52.333325 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37a3b9-6423-4518-a0d6-30d6c7ef3edc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:51:52.333934 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37a3b9-6423-4518-a0d6-30d6c7ef3edc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:52.335594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37a3b9-6423-4518-a0d6-30d6c7ef3edc None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:52.345640 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 14/27] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:51:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:51:52.352437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a6c10-f161-47b1-8ce3-7ec3f4b6b572 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:52.365341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37a3b9-6423-4518-a0d6-30d6c7ef3edc None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:52.474283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:52.522417 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37a3b9-6423-4518-a0d6-30d6c7ef3edc None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:52.535059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a6c10-f161-47b1-8ce3-7ec3f4b6b572 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:52.564782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12965cd4-9a0b-4b84-a07e-9c0759a239aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:52.565981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12965cd4-9a0b-4b84-a07e-9c0759a239aa None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:52.566484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12965cd4-9a0b-4b84-a07e-9c0759a239aa None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:52.567119 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12965cd4-9a0b-4b84-a07e-9c0759a239aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:52.580285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e38a35-0d6b-46f8-990f-2e7908d9596c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:52.581193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e38a35-0d6b-46f8-990f-2e7908d9596c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:52.581479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e38a35-0d6b-46f8-990f-2e7908d9596c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:52.581758 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e38a35-0d6b-46f8-990f-2e7908d9596c None admin] PATH_INFO: `/v3/users/baf04811d48d4c3fa638546a709d7179` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:52.583345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12965cd4-9a0b-4b84-a07e-9c0759a239aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:52.583722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e38a35-0d6b-46f8-990f-2e7908d9596c None admin] RBAC: Authorizing `identity:get_user(user_id=baf04811d48d4c3fa638546a709d7179)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:52.585208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e38a35-0d6b-46f8-990f-2e7908d9596c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:52.585585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12965cd4-9a0b-4b84-a07e-9c0759a239aa None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:52.586910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 14/28] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:51:52 2022] GET /identity/v3/users/baf04811d48d4c3fa638546a709d7179 => generated 247 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:51:52.588712 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cdc0f-8a7e-481f-8e8f-93640140c738 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:52.589004 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cdc0f-8a7e-481f-8e8f-93640140c738 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:52.589287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cdc0f-8a7e-481f-8e8f-93640140c738 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:52.591521 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 15/29] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:51:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:51:52.592888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee9a0c-f7c9-4132-bf10-828ac19cb9a9 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:52.593309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee9a0c-f7c9-4132-bf10-828ac19cb9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:52.593826 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee9a0c-f7c9-4132-bf10-828ac19cb9a9 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:52.595714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 16/30] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:51:52.596909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee9a0c-f7c9-4132-bf10-828ac19cb9a9 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:52.625549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-12965cd4-9a0b-4b84-a07e-9c0759a239aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:51:52.625549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:51:52.625549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:51:52.625549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:51:52.625549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:51:52.625549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:51:52.625549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:51:52.625549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:51:52.625549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:51:52.625549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:51:52.625549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:51:52.625549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:51:52.625549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:51:52.625549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:51:52.625549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:51:52.626466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:51:52.627310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:51:52.627310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:51:52.627310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:51:52.627310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:51:52.628886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 15/31] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:51:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:51:52.630450 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6546c9c-7332-4dab-a370-cb22b8b7a1f6 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:52.630869 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6546c9c-7332-4dab-a370-cb22b8b7a1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:52.631152 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6546c9c-7332-4dab-a370-cb22b8b7a1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:52.655982 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:52.659737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-c6546c9c-7332-4dab-a370-cb22b8b7a1f6 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:51:52.661462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c6546c9c-7332-4dab-a370-cb22b8b7a1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:51:52.722829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee9a0c-f7c9-4132-bf10-828ac19cb9a9 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:52.770565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 16/32] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:51:52.770565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1dc02-a7c4-45fc-b2b1-39df6164e41f None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:52.770565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1dc02-a7c4-45fc-b2b1-39df6164e41f None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:52.770565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1dc02-a7c4-45fc-b2b1-39df6164e41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:52.774293 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006acde-712f-4862-83a5-4fb54adaf6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:52.775194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006acde-712f-4862-83a5-4fb54adaf6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:52.775487 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006acde-712f-4862-83a5-4fb54adaf6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:52.775768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006acde-712f-4862-83a5-4fb54adaf6e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:52.779498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006acde-712f-4862-83a5-4fb54adaf6e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:52.781130 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006acde-712f-4862-83a5-4fb54adaf6e9 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:52.783020 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 17/33] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:51:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:51:52.783622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006acde-712f-4862-83a5-4fb54adaf6e9 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:52.798915 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-2fb1dc02-a7c4-45fc-b2b1-39df6164e41f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:51:52.800867 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-2fb1dc02-a7c4-45fc-b2b1-39df6164e41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:51:52.852164 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:52.881164 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 17/34] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:51:52.881539 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1dc02-a7c4-45fc-b2b1-39df6164e41f None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:52.912103 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006acde-712f-4862-83a5-4fb54adaf6e9 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:52.939501 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:52.963332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591559a-9953-4dc8-9a0b-cd40857a1a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:52.964631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591559a-9953-4dc8-9a0b-cd40857a1a95 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:52.965033 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591559a-9953-4dc8-9a0b-cd40857a1a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:52.965725 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591559a-9953-4dc8-9a0b-cd40857a1a95 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:52.966659 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591559a-9953-4dc8-9a0b-cd40857a1a95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:51:52.967348 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591559a-9953-4dc8-9a0b-cd40857a1a95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:52.969982 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591559a-9953-4dc8-9a0b-cd40857a1a95 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:52.990736 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 18/35] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:51:52.991848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591559a-9953-4dc8-9a0b-cd40857a1a95 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:53.044288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1dc02-a7c4-45fc-b2b1-39df6164e41f None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:53.083870 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b3691-0ad4-4da2-b296-e7fdf593ff4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:53.085190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b3691-0ad4-4da2-b296-e7fdf593ff4d None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:53.085684 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b3691-0ad4-4da2-b296-e7fdf593ff4d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:53.086170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b3691-0ad4-4da2-b296-e7fdf593ff4d None admin] PATH_INFO: `/v3/domains/default/users/baf04811d48d4c3fa638546a709d7179/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:53.088790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b3691-0ad4-4da2-b296-e7fdf593ff4d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=baf04811d48d4c3fa638546a709d7179, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:53.090590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b3691-0ad4-4da2-b296-e7fdf593ff4d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:53.132527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591559a-9953-4dc8-9a0b-cd40857a1a95 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:53.184519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2978e-50b6-46eb-b4de-2ecd20168bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:53.185210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2978e-50b6-46eb-b4de-2ecd20168bac None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:53.185518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2978e-50b6-46eb-b4de-2ecd20168bac None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:53.185795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2978e-50b6-46eb-b4de-2ecd20168bac None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:53.186590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2978e-50b6-46eb-b4de-2ecd20168bac None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:53.188873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2978e-50b6-46eb-b4de-2ecd20168bac None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:53.298332 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 18/36] 10.222.0.83 () {58 vars in 1361 bytes} [Mon Jul 18 14:51:52 2022] PUT /identity/v3/domains/default/users/baf04811d48d4c3fa638546a709d7179/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:51:53.299504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b3691-0ad4-4da2-b296-e7fdf593ff4d None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:53.351682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:53.397578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 19/37] 10.222.0.83 () {60 vars in 1131 bytes} [Mon Jul 18 14:51:52 2022] POST /identity/v3/roles => generated 219 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:51:53.398794 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2978e-50b6-46eb-b4de-2ecd20168bac None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:53.458697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b3691-0ad4-4da2-b296-e7fdf593ff4d None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:53.486466 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:53.519767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269898cd-3ba8-4da0-a614-44ee08a1ab4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:53.520421 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269898cd-3ba8-4da0-a614-44ee08a1ab4d None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:53.520717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269898cd-3ba8-4da0-a614-44ee08a1ab4d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:53.521007 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269898cd-3ba8-4da0-a614-44ee08a1ab4d None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:53.521913 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269898cd-3ba8-4da0-a614-44ee08a1ab4d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:53.523550 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269898cd-3ba8-4da0-a614-44ee08a1ab4d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:53.582387 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2978e-50b6-46eb-b4de-2ecd20168bac None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:53.623140 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 19/38] 10.222.0.83 () {60 vars in 1131 bytes} [Mon Jul 18 14:51:53 2022] POST /identity/v3/roles => generated 225 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:51:53.625457 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ddc0c-25ff-4037-8948-59d0dc064af4 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:53.625457 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ddc0c-25ff-4037-8948-59d0dc064af4 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:53.625962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ddc0c-25ff-4037-8948-59d0dc064af4 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:53.627468 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 20/39] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:51:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:51:53.628692 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a2cde-746b-49cb-96ff-f490a5a88d31 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:53.628982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a2cde-746b-49cb-96ff-f490a5a88d31 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:53.629263 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a2cde-746b-49cb-96ff-f490a5a88d31 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:53.634585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 21/40] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:51:53.634585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b8c71-5872-4b05-94db-7c437fee2d43 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:53.634585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b8c71-5872-4b05-94db-7c437fee2d43 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:53.634585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b8c71-5872-4b05-94db-7c437fee2d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:53.645763 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cdb7e-b014-4e36-a14f-cdc75bc6b16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:53.646891 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cdb7e-b014-4e36-a14f-cdc75bc6b16e None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:53.647368 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cdb7e-b014-4e36-a14f-cdc75bc6b16e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:53.647830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cdb7e-b014-4e36-a14f-cdc75bc6b16e None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:53.648833 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3cdb7e-b014-4e36-a14f-cdc75bc6b16e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:53.652079 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3cdb7e-b014-4e36-a14f-cdc75bc6b16e None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:53.662749 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-191b8c71-5872-4b05-94db-7c437fee2d43 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:51:53.664455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-191b8c71-5872-4b05-94db-7c437fee2d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:51:53.786957 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 20/41] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Jul 18 14:51:53 2022] POST /identity/v3/projects => generated 301 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:51:53.789321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c1ac8-4925-4c6b-be28-45527b2c2b7e None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:53.789898 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c1ac8-4925-4c6b-be28-45527b2c2b7e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:53.790371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c1ac8-4925-4c6b-be28-45527b2c2b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:53.829547 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-142c1ac8-4925-4c6b-be28-45527b2c2b7e None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:51:53.835320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-142c1ac8-4925-4c6b-be28-45527b2c2b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:51:53.891920 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 22/42] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:51:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:51:53.920230 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b8c71-5872-4b05-94db-7c437fee2d43 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:53.955182 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 21/43] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:51:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:51:53.964365 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:53.985971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c1ac8-4925-4c6b-be28-45527b2c2b7e None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:54.004586 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:54.069541 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b8c71-5872-4b05-94db-7c437fee2d43 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:54.115543 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124cedc-0e23-4f26-a50f-542cce406f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:54.116449 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124cedc-0e23-4f26-a50f-542cce406f8a None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:54.116896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124cedc-0e23-4f26-a50f-542cce406f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:54.117180 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124cedc-0e23-4f26-a50f-542cce406f8a None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:54.118012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124cedc-0e23-4f26-a50f-542cce406f8a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:54.118979 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c1ac8-4925-4c6b-be28-45527b2c2b7e None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:54.119453 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124cedc-0e23-4f26-a50f-542cce406f8a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:54.156373 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577c17c-cfd1-4527-8f36-9317a8d65586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:54.157187 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577c17c-cfd1-4527-8f36-9317a8d65586 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:54.157478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577c17c-cfd1-4527-8f36-9317a8d65586 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:54.157763 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577c17c-cfd1-4527-8f36-9317a8d65586 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:54.159022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577c17c-cfd1-4527-8f36-9317a8d65586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:54.160507 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577c17c-cfd1-4527-8f36-9317a8d65586 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:54.162159 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 22/44] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:51:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:51:54.167936 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577c17c-cfd1-4527-8f36-9317a8d65586 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:54.192949 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 23/45] 10.222.0.83 () {60 vars in 1131 bytes} [Mon Jul 18 14:51:53 2022] POST /identity/v3/roles => generated 223 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:51:54.274490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:54.296772 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577c17c-cfd1-4527-8f36-9317a8d65586 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:54.335099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012155f-6228-4682-810f-d58ab41d2e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:54.335810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012155f-6228-4682-810f-d58ab41d2e54 None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:54.336101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012155f-6228-4682-810f-d58ab41d2e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:54.336385 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012155f-6228-4682-810f-d58ab41d2e54 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:54.337118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012155f-6228-4682-810f-d58ab41d2e54 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:54.338732 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012155f-6228-4682-810f-d58ab41d2e54 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:54.422691 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 23/46] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Jul 18 14:51:54 2022] POST /identity/v3/projects => generated 312 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:51:55.603958 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42a932-7452-43c4-8065-fde2e9845d4e None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:55.604509 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42a932-7452-43c4-8065-fde2e9845d4e None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:55.604509 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42a932-7452-43c4-8065-fde2e9845d4e None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:55.605957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 24/47] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:51:55.612677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9c51e-03df-4a1b-b3b8-d305ca826d7f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:55.613141 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9c51e-03df-4a1b-b3b8-d305ca826d7f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:55.613349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9c51e-03df-4a1b-b3b8-d305ca826d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:55.632729 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-21e9c51e-03df-4a1b-b3b8-d305ca826d7f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:51:55.633978 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-21e9c51e-03df-4a1b-b3b8-d305ca826d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:51:55.704097 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 24/48] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:51:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:51:55.727134 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42a932-7452-43c4-8065-fde2e9845d4e None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:55.824392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:55.837995 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42a932-7452-43c4-8065-fde2e9845d4e None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:55.872354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ee58d-0403-46da-96f9-f93fea27d8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:55.873123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ee58d-0403-46da-96f9-f93fea27d8cc None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:55.873407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ee58d-0403-46da-96f9-f93fea27d8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:55.873714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ee58d-0403-46da-96f9-f93fea27d8cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:55.885064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ee58d-0403-46da-96f9-f93fea27d8cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:51:55.893484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-bc6ee58d-0403-46da-96f9-f93fea27d8cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:51:55.893484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:51:55.893484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:51:55.893484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:51:55.893484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:51:55.893484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:51:55.893484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:51:55.893484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:51:55.893484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:51:55.893484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:51:55.893484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:51:55.893484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:51:55.893484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:51:55.893484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:51:55.893484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:51:55.893484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:51:55.894402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:51:55.894402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:51:55.894402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:51:55.894402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:51:55.894402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:51:55.894402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:51:55.894402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:51:55.894402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:51:55.894402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:51:55.894402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:51:55.894402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:51:55.894402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:51:55.894402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:51:55.894402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:51:55.894402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:51:55.894402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:51:55.895219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:51:55.895219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:51:55.895219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:51:55.895219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:51:55.895219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:51:55.895219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:51:55.895219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:51:55.895219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:51:55.895219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:51:55.895219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:51:55.895219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:51:55.895219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:51:55.895219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 25/49] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:51:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:51:55.900726 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9c51e-03df-4a1b-b3b8-d305ca826d7f None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:55.963292 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:56.003736 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9c51e-03df-4a1b-b3b8-d305ca826d7f None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:56.042329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb4c9a-fe9f-42fb-b72a-759f2a4ebc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:56.043044 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb4c9a-fe9f-42fb-b72a-759f2a4ebc26 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:56.043340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb4c9a-fe9f-42fb-b72a-759f2a4ebc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:56.043632 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb4c9a-fe9f-42fb-b72a-759f2a4ebc26 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:56.044401 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb4c9a-fe9f-42fb-b72a-759f2a4ebc26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:51:56.044719 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb4c9a-fe9f-42fb-b72a-759f2a4ebc26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:56.046137 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb4c9a-fe9f-42fb-b72a-759f2a4ebc26 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:56.058778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 25/50] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:51:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:51:56.064912 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ee58d-0403-46da-96f9-f93fea27d8cc None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:56.166001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ee58d-0403-46da-96f9-f93fea27d8cc None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:56.199719 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042bc68-a198-48bd-96c9-2157e03a642d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:56.200623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042bc68-a198-48bd-96c9-2157e03a642d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:56.200911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042bc68-a198-48bd-96c9-2157e03a642d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:56.201190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042bc68-a198-48bd-96c9-2157e03a642d None admin] PATH_INFO: `/v3/users/baf04811d48d4c3fa638546a709d7179` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:56.202403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042bc68-a198-48bd-96c9-2157e03a642d None admin] RBAC: Authorizing `identity:get_user(user_id=baf04811d48d4c3fa638546a709d7179)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:56.204778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042bc68-a198-48bd-96c9-2157e03a642d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:56.206446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 26/51] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:51:56 2022] GET /identity/v3/users/baf04811d48d4c3fa638546a709d7179 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:51:56.211941 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb4c9a-fe9f-42fb-b72a-759f2a4ebc26 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:56.320989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb4c9a-fe9f-42fb-b72a-759f2a4ebc26 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:56.459417 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:56.467882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82373fa5-322d-48a8-b953-1e5a7b0eee39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:56.471306 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82373fa5-322d-48a8-b953-1e5a7b0eee39 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:56.471306 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82373fa5-322d-48a8-b953-1e5a7b0eee39 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:56.471306 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82373fa5-322d-48a8-b953-1e5a7b0eee39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:56.471306 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82373fa5-322d-48a8-b953-1e5a7b0eee39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:56.472533 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82373fa5-322d-48a8-b953-1e5a7b0eee39 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:56.474165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 26/52] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:51:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:51:56.480173 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042bc68-a198-48bd-96c9-2157e03a642d None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:56.501271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:56.518103 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93077606-b707-4309-b744-8bcdc8a69df4 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:56.518404 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93077606-b707-4309-b744-8bcdc8a69df4 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:56.518718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93077606-b707-4309-b744-8bcdc8a69df4 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:56.520097 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 27/53] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:51:56.525309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c48b66-b218-4f9e-bee1-53c07c445d55 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:56.525600 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c48b66-b218-4f9e-bee1-53c07c445d55 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:56.525882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c48b66-b218-4f9e-bee1-53c07c445d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:56.543541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d1c48b66-b218-4f9e-bee1-53c07c445d55 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:51:56.544759 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d1c48b66-b218-4f9e-bee1-53c07c445d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:51:56.586571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042bc68-a198-48bd-96c9-2157e03a642d None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:56.617755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 28/54] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:51:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:51:56.624008 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf6a89-9d84-4375-a1ff-4b16a74057da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:56.624739 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf6a89-9d84-4375-a1ff-4b16a74057da None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:56.625031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf6a89-9d84-4375-a1ff-4b16a74057da None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:56.625487 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf6a89-9d84-4375-a1ff-4b16a74057da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:56.626344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf6a89-9d84-4375-a1ff-4b16a74057da None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.domain.id=default, user.id=baf04811d48d4c3fa638546a709d7179) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:51:56.626795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf6a89-9d84-4375-a1ff-4b16a74057da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:56.628284 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf6a89-9d84-4375-a1ff-4b16a74057da None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:56.634167 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 27/55] 10.222.0.83 () {56 vars in 1322 bytes} [Mon Jul 18 14:51:56 2022] GET /identity/v3/role_assignments?user.id=baf04811d48d4c3fa638546a709d7179&scope.domain.id=default&role.id=5891920310b54787bd1152d736d2a3f1 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:51:56.642248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c48b66-b218-4f9e-bee1-53c07c445d55 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:56.754465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c48b66-b218-4f9e-bee1-53c07c445d55 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:56.784234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:56.797809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3205f-9bb6-466e-800c-506476d5bb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:56.798657 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3205f-9bb6-466e-800c-506476d5bb46 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:56.798949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3205f-9bb6-466e-800c-506476d5bb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:56.799229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3205f-9bb6-466e-800c-506476d5bb46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:56.801329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3205f-9bb6-466e-800c-506476d5bb46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:56.802840 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3205f-9bb6-466e-800c-506476d5bb46 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:56.804516 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 29/56] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:51:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:51:56.811165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf6a89-9d84-4375-a1ff-4b16a74057da None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:56.897563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:56.917066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf6a89-9d84-4375-a1ff-4b16a74057da None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:56.951473 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f890e1-afe0-4296-a0de-01160b07c627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:56.952149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f890e1-afe0-4296-a0de-01160b07c627 None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:56.952435 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f890e1-afe0-4296-a0de-01160b07c627 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:56.953294 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f890e1-afe0-4296-a0de-01160b07c627 None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:56.954133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f890e1-afe0-4296-a0de-01160b07c627 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:56.955667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f890e1-afe0-4296-a0de-01160b07c627 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:57.039663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 28/57] 10.222.0.83 () {60 vars in 1140 bytes} [Mon Jul 18 14:51:56 2022] POST /identity/v3/projects => generated 298 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:51:59.114373 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45050555-ebe4-491c-9247-080b476bf787 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:59.114983 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45050555-ebe4-491c-9247-080b476bf787 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:59.114983 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45050555-ebe4-491c-9247-080b476bf787 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:59.116362 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 30/58] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:51:59.121708 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c8048-3f5c-4111-bd25-0df5891b8095 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:59.122137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c8048-3f5c-4111-bd25-0df5891b8095 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:59.122287 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c8048-3f5c-4111-bd25-0df5891b8095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:59.140530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-c25c8048-3f5c-4111-bd25-0df5891b8095 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:51:59.141881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c25c8048-3f5c-4111-bd25-0df5891b8095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:51:59.212874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 29/59] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:51:59.237791 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45050555-ebe4-491c-9247-080b476bf787 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:59.340542 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45050555-ebe4-491c-9247-080b476bf787 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:59.376792 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149a529-a07e-41ec-907f-4b9d1aa1ac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:59.377580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149a529-a07e-41ec-907f-4b9d1aa1ac87 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:59.377866 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149a529-a07e-41ec-907f-4b9d1aa1ac87 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:59.378145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149a529-a07e-41ec-907f-4b9d1aa1ac87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:59.379436 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149a529-a07e-41ec-907f-4b9d1aa1ac87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:59.380884 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149a529-a07e-41ec-907f-4b9d1aa1ac87 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:59.382509 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 31/60] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:51:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:51:59.388518 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c8048-3f5c-4111-bd25-0df5891b8095 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:51:59.455049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:51:59.494113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c8048-3f5c-4111-bd25-0df5891b8095 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:51:59.528879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8840b6-1446-4d88-b5a4-12f77a5f56c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:51:59.529551 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8840b6-1446-4d88-b5a4-12f77a5f56c8 None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:51:59.529840 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8840b6-1446-4d88-b5a4-12f77a5f56c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:51:59.530118 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8840b6-1446-4d88-b5a4-12f77a5f56c8 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:51:59.531485 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8840b6-1446-4d88-b5a4-12f77a5f56c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:51:59.533073 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8840b6-1446-4d88-b5a4-12f77a5f56c8 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:51:59.664294 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 30/61] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:51:59 2022] POST /identity/v3/users => generated 275 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:52:02.919226 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a59e1d-0206-474e-a6ac-c7546a555f8f None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:02.920441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a59e1d-0206-474e-a6ac-c7546a555f8f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:02.926490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a59e1d-0206-474e-a6ac-c7546a555f8f None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:02.929653 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 32/62] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:02 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:02.934222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e8062-ca94-440c-ac4c-e80b009f58d9 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:02.935198 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e8062-ca94-440c-ac4c-e80b009f58d9 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:02.935712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e8062-ca94-440c-ac4c-e80b009f58d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:02.964563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-b99e8062-ca94-440c-ac4c-e80b009f58d9 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:02.966232 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b99e8062-ca94-440c-ac4c-e80b009f58d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:03.054380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 31/63] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:03.097677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a59e1d-0206-474e-a6ac-c7546a555f8f None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:03.236021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:03.246483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a59e1d-0206-474e-a6ac-c7546a555f8f None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:03.313498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18280387-cfd6-458f-969e-43c5ec4e12e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:03.314329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18280387-cfd6-458f-969e-43c5ec4e12e0 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:03.314647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18280387-cfd6-458f-969e-43c5ec4e12e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:03.314938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18280387-cfd6-458f-969e-43c5ec4e12e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:03.340629 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d34488-1b98-4c38-b76d-492eb7f32964 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:03.340932 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d34488-1b98-4c38-b76d-492eb7f32964 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:03.341236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d34488-1b98-4c38-b76d-492eb7f32964 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:03.343801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 32/64] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:03.344668 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18280387-cfd6-458f-969e-43c5ec4e12e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:03.356836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-18280387-cfd6-458f-969e-43c5ec4e12e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:03.356836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:03.356836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:03.356836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:03.356836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:03.356836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:03.356836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:03.356836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:03.356836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:03.356836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:03.356836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:03.356836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:03.356836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:03.356836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:03.356836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:03.356836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:03.358110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:03.358110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:03.358110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:03.358110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:03.358110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:03.358110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:03.358110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:03.358110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:03.358110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:03.358110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:03.358110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:03.358110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:03.358110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:03.358110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:03.358110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:03.358110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:03.359778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:03.359778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:03.359778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:03.359778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:03.359778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:03.359778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:03.359778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:03.359778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:03.359778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:03.359778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:03.359778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:03.359778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:03.359778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 33/65] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:03.368359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d34488-1b98-4c38-b76d-492eb7f32964 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:03.380346 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a4e2e-5698-4f8a-94a6-d5c92c2f95b4 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:03.380681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a4e2e-5698-4f8a-94a6-d5c92c2f95b4 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:03.380905 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a4e2e-5698-4f8a-94a6-d5c92c2f95b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:03.412818 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d98a4e2e-5698-4f8a-94a6-d5c92c2f95b4 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:03.414041 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d98a4e2e-5698-4f8a-94a6-d5c92c2f95b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:03.466982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:03.499974 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 34/66] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:03.500462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d34488-1b98-4c38-b76d-492eb7f32964 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:03.501307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512a3a5-b528-480d-ba3a-31309dc053a1 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:03.501609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512a3a5-b528-480d-ba3a-31309dc053a1 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:03.501895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512a3a5-b528-480d-ba3a-31309dc053a1 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:03.503682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 35/67] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:03.513251 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041af2a-ee26-4a90-8a58-95bee01a7fbe None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:03.513577 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041af2a-ee26-4a90-8a58-95bee01a7fbe None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:03.513812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041af2a-ee26-4a90-8a58-95bee01a7fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:03.533111 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-5041af2a-ee26-4a90-8a58-95bee01a7fbe None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:03.534347 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-5041af2a-ee26-4a90-8a58-95bee01a7fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:03.540285 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a4940-6b57-4a56-9276-0505d5aa4c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:03.540656 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a4940-6b57-4a56-9276-0505d5aa4c91 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:03.540950 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a4940-6b57-4a56-9276-0505d5aa4c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:03.541239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a4940-6b57-4a56-9276-0505d5aa4c91 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:03.542279 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a4940-6b57-4a56-9276-0505d5aa4c91 None admin] RBAC: Adding query filter params (name=member) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:03.542480 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a4940-6b57-4a56-9276-0505d5aa4c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:03.543837 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a4940-6b57-4a56-9276-0505d5aa4c91 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:03.559196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 33/68] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:03.559196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d20fb2-485e-4675-b0f5-87995a2010ae None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:03.559196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d20fb2-485e-4675-b0f5-87995a2010ae None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:03.559196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d20fb2-485e-4675-b0f5-87995a2010ae None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:03.559196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 34/69] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:03.559196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d20fb2-485e-4675-b0f5-87995a2010ae None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:03.596957 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:03.621048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 36/70] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:03.621928 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041af2a-ee26-4a90-8a58-95bee01a7fbe None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:03.707550 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d20fb2-485e-4675-b0f5-87995a2010ae None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:03.726417 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:03.756706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c7278d-4196-4713-984d-322a213b1f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:03.757423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7278d-4196-4713-984d-322a213b1f64 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:03.757711 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7278d-4196-4713-984d-322a213b1f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:03.757993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7278d-4196-4713-984d-322a213b1f64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:03.769199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041af2a-ee26-4a90-8a58-95bee01a7fbe None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:03.770232 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c7278d-4196-4713-984d-322a213b1f64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:03.778694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-d7c7278d-4196-4713-984d-322a213b1f64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:03.778694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:03.778694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:03.778694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:03.778694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:03.778694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:03.778694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:03.778694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:03.778694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:03.778694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:03.778694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:03.778694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:03.778694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:03.778694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:03.778694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:03.778694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:03.779621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:03.779621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:03.779621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:03.779621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:03.779621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:03.779621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:03.779621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:03.779621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:03.779621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:03.779621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:03.779621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:03.779621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:03.779621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:03.779621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:03.779621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:03.779621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:03.780997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:03.780997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:03.780997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:03.780997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:03.780997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:03.780997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:03.780997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:03.780997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:03.780997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:03.780997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:03.780997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:03.780997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:03.780997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 35/71] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:52:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:03.782686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438f62f-82c7-43a3-8e75-e3b168884b77 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:03.782686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438f62f-82c7-43a3-8e75-e3b168884b77 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:03.782686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438f62f-82c7-43a3-8e75-e3b168884b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:03.800957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-b438f62f-82c7-43a3-8e75-e3b168884b77 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:03.802316 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b438f62f-82c7-43a3-8e75-e3b168884b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:03.813106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be53e5c-4481-4c55-b1d2-ab1fb9a9b4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:03.814002 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be53e5c-4481-4c55-b1d2-ab1fb9a9b4ab None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:03.814292 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be53e5c-4481-4c55-b1d2-ab1fb9a9b4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:03.814608 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be53e5c-4481-4c55-b1d2-ab1fb9a9b4ab None admin] PATH_INFO: `/v3/users/b7bfc576d3154257a676f91304b0d7b4` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:03.831705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be53e5c-4481-4c55-b1d2-ab1fb9a9b4ab None admin] RBAC: Authorizing `identity:get_user(user_id=b7bfc576d3154257a676f91304b0d7b4)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:03.833217 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be53e5c-4481-4c55-b1d2-ab1fb9a9b4ab None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:03.836292 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 37/72] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:03 2022] GET /identity/v3/users/b7bfc576d3154257a676f91304b0d7b4 => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:03.839456 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be53e5c-4481-4c55-b1d2-ab1fb9a9b4ab None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:03.883492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 36/73] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:03.884397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438f62f-82c7-43a3-8e75-e3b168884b77 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:03.886016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:03.962704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be53e5c-4481-4c55-b1d2-ab1fb9a9b4ab None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:03.993690 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:03.998369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f03b17-915a-41b2-bc68-609415d22788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:03.998729 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438f62f-82c7-43a3-8e75-e3b168884b77 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:03.999381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f03b17-915a-41b2-bc68-609415d22788 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:03.999651 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f03b17-915a-41b2-bc68-609415d22788 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:04.000134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f03b17-915a-41b2-bc68-609415d22788 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:04.010983 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f03b17-915a-41b2-bc68-609415d22788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:04.020279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-04f03b17-915a-41b2-bc68-609415d22788 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:04.020279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:04.020279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:04.020279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:04.020279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:04.020279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:04.020279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:04.020279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:04.020279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:04.020279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:04.020279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:04.020279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:04.020279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:04.020279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:04.020279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:04.020279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:04.021411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:04.021411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:04.021411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:04.021411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:04.021411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:04.021411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:04.021411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:04.021411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:04.021411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:04.021411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:04.021411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:04.021411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:04.021411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:04.021411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:04.021411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:04.021411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:04.022398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:04.022398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:04.022398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:04.022398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:04.022398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:04.022398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:04.022398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:04.022398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:04.022398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:04.022398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:04.022398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:04.022398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:04.022398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 38/74] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:52:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:04.022398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f03b17-915a-41b2-bc68-609415d22788 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:04.044406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2df05f-2494-467e-912a-c34569ca8639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:04.045091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2df05f-2494-467e-912a-c34569ca8639 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:04.045387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2df05f-2494-467e-912a-c34569ca8639 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:04.045670 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2df05f-2494-467e-912a-c34569ca8639 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:04.046851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2df05f-2494-467e-912a-c34569ca8639 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:04.047156 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2df05f-2494-467e-912a-c34569ca8639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:04.048601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2df05f-2494-467e-912a-c34569ca8639 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:04.057303 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 37/75] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:04.058627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c4ece-1f41-4437-82e9-06fefe1f6425 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:04.058924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c4ece-1f41-4437-82e9-06fefe1f6425 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:04.059291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c4ece-1f41-4437-82e9-06fefe1f6425 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:04.060566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 38/76] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:04.061267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c4ece-1f41-4437-82e9-06fefe1f6425 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:04.138289 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:04.142569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f03b17-915a-41b2-bc68-609415d22788 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:04.178258 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c4ece-1f41-4437-82e9-06fefe1f6425 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:04.179863 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa9a12-f241-464d-9d0b-47baf1fb0eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:04.180544 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa9a12-f241-464d-9d0b-47baf1fb0eca None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:04.180846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa9a12-f241-464d-9d0b-47baf1fb0eca None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:04.181136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa9a12-f241-464d-9d0b-47baf1fb0eca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:04.216014 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2842072-7f1f-478b-9103-0956033bc6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:04.216838 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2842072-7f1f-478b-9103-0956033bc6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:04.217130 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2842072-7f1f-478b-9103-0956033bc6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:04.217413 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2842072-7f1f-478b-9103-0956033bc6b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:04.228947 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2842072-7f1f-478b-9103-0956033bc6b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:04.237345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-f2842072-7f1f-478b-9103-0956033bc6b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:04.237345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:04.237345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:04.237345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:04.237345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:04.237345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:04.237345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:04.237345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:04.237345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:04.237345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:04.237345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:04.237345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:04.237345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:04.237345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:04.237345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:04.237345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:04.240914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:04.240914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:04.240914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:04.240914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:04.240914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:04.240914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:04.240914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:04.240914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:04.240914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:04.240914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:04.240914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:04.240914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:04.240914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:04.240914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:04.240914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:04.240914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:04.242012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:04.242012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:04.242012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:04.242012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:04.242012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:04.242012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:04.242012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:04.242012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:04.242012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:04.242012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:04.242012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:04.242012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:04.242012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 39/77] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:04.242012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2842072-7f1f-478b-9103-0956033bc6b9 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:04.262358 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa9a12-f241-464d-9d0b-47baf1fb0eca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:04.263831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa9a12-f241-464d-9d0b-47baf1fb0eca None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:04.266829 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:04.352199 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2842072-7f1f-478b-9103-0956033bc6b9 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:04.364105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:04.368745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 39/78] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:04.369589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa9a12-f241-464d-9d0b-47baf1fb0eca None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:04.398345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f03cd-136f-45e3-a7d7-44a9e95f52d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:04.399055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f03cd-136f-45e3-a7d7-44a9e95f52d6 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:04.399354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f03cd-136f-45e3-a7d7-44a9e95f52d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:04.399635 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f03cd-136f-45e3-a7d7-44a9e95f52d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:04.400392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f03cd-136f-45e3-a7d7-44a9e95f52d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:04.400706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f03cd-136f-45e3-a7d7-44a9e95f52d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:04.402126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f03cd-136f-45e3-a7d7-44a9e95f52d6 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:04.410441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 40/79] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:52:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:04.411808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c3a3b-6d0f-4fd3-a5d3-e9cb0003324e None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:04.412110 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c3a3b-6d0f-4fd3-a5d3-e9cb0003324e None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:04.412393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c3a3b-6d0f-4fd3-a5d3-e9cb0003324e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:04.429328 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-b32c3a3b-6d0f-4fd3-a5d3-e9cb0003324e None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:04.430580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b32c3a3b-6d0f-4fd3-a5d3-e9cb0003324e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:04.479646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa9a12-f241-464d-9d0b-47baf1fb0eca None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:04.501354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 41/80] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:04.502201 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c3a3b-6d0f-4fd3-a5d3-e9cb0003324e None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:04.514067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451b988-a2ee-4868-9e07-57d23e656232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:04.515541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451b988-a2ee-4868-9e07-57d23e656232 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:04.515835 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451b988-a2ee-4868-9e07-57d23e656232 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:04.516122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451b988-a2ee-4868-9e07-57d23e656232 None admin] PATH_INFO: `/v3/users/baf04811d48d4c3fa638546a709d7179` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:04.517447 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451b988-a2ee-4868-9e07-57d23e656232 None admin] RBAC: Authorizing `identity:get_user(user_id=baf04811d48d4c3fa638546a709d7179)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:04.518942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451b988-a2ee-4868-9e07-57d23e656232 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:04.519924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:04.520961 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 40/81] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:04 2022] GET /identity/v3/users/baf04811d48d4c3fa638546a709d7179 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:04.521366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451b988-a2ee-4868-9e07-57d23e656232 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:04.588738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:04.611452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c3a3b-6d0f-4fd3-a5d3-e9cb0003324e None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:04.626249 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451b988-a2ee-4868-9e07-57d23e656232 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:04.646094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb867d-4c48-47ee-8687-30fc0b5a12b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:04.646813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb867d-4c48-47ee-8687-30fc0b5a12b5 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:04.647120 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb867d-4c48-47ee-8687-30fc0b5a12b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:04.647389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb867d-4c48-47ee-8687-30fc0b5a12b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:04.648147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb867d-4c48-47ee-8687-30fc0b5a12b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:04.648459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb867d-4c48-47ee-8687-30fc0b5a12b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:04.649887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb867d-4c48-47ee-8687-30fc0b5a12b5 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:04.659065 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 42/82] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:04.659972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb867d-4c48-47ee-8687-30fc0b5a12b5 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:04.661637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715d7e0-e7e9-42ee-b207-253d32d2099c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:04.662516 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715d7e0-e7e9-42ee-b207-253d32d2099c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:04.662829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715d7e0-e7e9-42ee-b207-253d32d2099c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:04.663108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715d7e0-e7e9-42ee-b207-253d32d2099c None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:04.664708 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715d7e0-e7e9-42ee-b207-253d32d2099c None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:04.666160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715d7e0-e7e9-42ee-b207-253d32d2099c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:04.667907 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 41/83] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:04 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:04.668671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715d7e0-e7e9-42ee-b207-253d32d2099c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:04.743491 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:04.767627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb867d-4c48-47ee-8687-30fc0b5a12b5 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:04.773894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715d7e0-e7e9-42ee-b207-253d32d2099c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:04.804060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379cec1-0b96-488e-9831-29f72e22c43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:04.804918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379cec1-0b96-488e-9831-29f72e22c43d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:04.805205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379cec1-0b96-488e-9831-29f72e22c43d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:04.805486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379cec1-0b96-488e-9831-29f72e22c43d None admin] PATH_INFO: `/v3/users/b7bfc576d3154257a676f91304b0d7b4` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:04.806723 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379cec1-0b96-488e-9831-29f72e22c43d None admin] RBAC: Authorizing `identity:get_user(user_id=b7bfc576d3154257a676f91304b0d7b4)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:04.808207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379cec1-0b96-488e-9831-29f72e22c43d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:04.809093 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577f4f9-6d80-4fb5-82ed-6e11d36a6a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:04.809741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577f4f9-6d80-4fb5-82ed-6e11d36a6a72 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:04.809953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 43/84] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:04 2022] GET /identity/v3/users/b7bfc576d3154257a676f91304b0d7b4 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:04.810047 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577f4f9-6d80-4fb5-82ed-6e11d36a6a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:04.810304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577f4f9-6d80-4fb5-82ed-6e11d36a6a72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:04.810756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379cec1-0b96-488e-9831-29f72e22c43d None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:04.817356 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:04.883543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8577f4f9-6d80-4fb5-82ed-6e11d36a6a72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:04.885019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8577f4f9-6d80-4fb5-82ed-6e11d36a6a72 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:04.912833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379cec1-0b96-488e-9831-29f72e22c43d None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:04.948106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73d0cc-edcd-483d-95c4-59bdd5fc64be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:04.948916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73d0cc-edcd-483d-95c4-59bdd5fc64be None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:04.949212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73d0cc-edcd-483d-95c4-59bdd5fc64be None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:04.949499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73d0cc-edcd-483d-95c4-59bdd5fc64be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:04.950725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73d0cc-edcd-483d-95c4-59bdd5fc64be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:04.952864 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73d0cc-edcd-483d-95c4-59bdd5fc64be None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:04.954635 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 44/85] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:52:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:04.955413 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73d0cc-edcd-483d-95c4-59bdd5fc64be None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:04.974558 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:04.991436 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 42/86] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:04.992303 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577f4f9-6d80-4fb5-82ed-6e11d36a6a72 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:05.028149 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:05.063266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73d0cc-edcd-483d-95c4-59bdd5fc64be None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:05.098136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577f4f9-6d80-4fb5-82ed-6e11d36a6a72 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:05.098687 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c765134-b165-4815-8c85-064a056081ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:05.099584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c765134-b165-4815-8c85-064a056081ea None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:05.099873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c765134-b165-4815-8c85-064a056081ea None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:05.100154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c765134-b165-4815-8c85-064a056081ea None admin] PATH_INFO: `/v3/users/b7bfc576d3154257a676f91304b0d7b4` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:05.101376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c765134-b165-4815-8c85-064a056081ea None admin] RBAC: Authorizing `identity:get_user(user_id=b7bfc576d3154257a676f91304b0d7b4)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:05.102906 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c765134-b165-4815-8c85-064a056081ea None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:05.104574 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 45/87] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:04 2022] GET /identity/v3/users/b7bfc576d3154257a676f91304b0d7b4 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:05.105350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c765134-b165-4815-8c85-064a056081ea None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:05.138658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee986788-5123-4598-9789-26ba4108d56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:05.140228 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee986788-5123-4598-9789-26ba4108d56e None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:05.140751 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee986788-5123-4598-9789-26ba4108d56e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:05.141262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee986788-5123-4598-9789-26ba4108d56e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:05.142156 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee986788-5123-4598-9789-26ba4108d56e None admin] RBAC: Adding query filter params (role.id=3ab0ee91b3284a65a0dd5696a6250883, scope.project.id=10cc8d215a4045f1b83e4120a718db23, user.id=b7bfc576d3154257a676f91304b0d7b4) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:05.142759 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee986788-5123-4598-9789-26ba4108d56e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:05.145573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee986788-5123-4598-9789-26ba4108d56e None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:05.154507 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 43/88] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:04 2022] GET /identity/v3/role_assignments?user.id=b7bfc576d3154257a676f91304b0d7b4&scope.project.id=10cc8d215a4045f1b83e4120a718db23&role.id=3ab0ee91b3284a65a0dd5696a6250883 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:05.155373 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee986788-5123-4598-9789-26ba4108d56e None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:05.217695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:05.240507 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c765134-b165-4815-8c85-064a056081ea None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:05.274402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:05.279681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee986788-5123-4598-9789-26ba4108d56e None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:05.282525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ce8fc-cb7e-45f0-abd9-332c5ada7e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:05.284360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ce8fc-cb7e-45f0-abd9-332c5ada7e06 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:05.284360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ce8fc-cb7e-45f0-abd9-332c5ada7e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:05.284360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ce8fc-cb7e-45f0-abd9-332c5ada7e06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:05.324648 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3264e68-cac1-4586-88c2-a46ca2c09eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:05.325009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3264e68-cac1-4586-88c2-a46ca2c09eb8 None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:05.325303 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3264e68-cac1-4586-88c2-a46ca2c09eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:05.326363 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3264e68-cac1-4586-88c2-a46ca2c09eb8 None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:05.334584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3264e68-cac1-4586-88c2-a46ca2c09eb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:05.334584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3264e68-cac1-4586-88c2-a46ca2c09eb8 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:05.362383 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ce8fc-cb7e-45f0-abd9-332c5ada7e06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:05.365580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ce8fc-cb7e-45f0-abd9-332c5ada7e06 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:05.418057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 44/89] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:52:05 2022] POST /identity/v3/users => generated 289 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:52:05.425968 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3264e68-cac1-4586-88c2-a46ca2c09eb8 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:05.468342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:05.494052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 46/90] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:05.498562 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ce8fc-cb7e-45f0-abd9-332c5ada7e06 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:05.592786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3264e68-cac1-4586-88c2-a46ca2c09eb8 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:05.632021 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ce8fc-cb7e-45f0-abd9-332c5ada7e06 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:05.647823 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327d9ad-6421-41d0-8fbb-f2d659190e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:05.649005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327d9ad-6421-41d0-8fbb-f2d659190e65 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:05.649293 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327d9ad-6421-41d0-8fbb-f2d659190e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:05.649572 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327d9ad-6421-41d0-8fbb-f2d659190e65 None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:05.651031 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327d9ad-6421-41d0-8fbb-f2d659190e65 None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:05.652943 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327d9ad-6421-41d0-8fbb-f2d659190e65 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:05.654703 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 45/91] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:05 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:05.655562 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327d9ad-6421-41d0-8fbb-f2d659190e65 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:05.686142 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:05.690755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147a815-ecee-4422-bd7d-70b9067b95de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:05.691316 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147a815-ecee-4422-bd7d-70b9067b95de None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:05.692364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147a815-ecee-4422-bd7d-70b9067b95de None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:05.692364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147a815-ecee-4422-bd7d-70b9067b95de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:05.752224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:05.788598 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147a815-ecee-4422-bd7d-70b9067b95de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:05.790690 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147a815-ecee-4422-bd7d-70b9067b95de None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:05.805340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327d9ad-6421-41d0-8fbb-f2d659190e65 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:05.859620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83c44c-7e68-40ae-b587-fd918b21c62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:05.860647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83c44c-7e68-40ae-b587-fd918b21c62d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:05.860984 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83c44c-7e68-40ae-b587-fd918b21c62d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:05.861089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83c44c-7e68-40ae-b587-fd918b21c62d None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:05.862853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83c44c-7e68-40ae-b587-fd918b21c62d None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:05.864506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83c44c-7e68-40ae-b587-fd918b21c62d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:05.866255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 46/92] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:05 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:05.867447 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83c44c-7e68-40ae-b587-fd918b21c62d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:05.943328 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 47/93] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:05.944202 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147a815-ecee-4422-bd7d-70b9067b95de None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:06.002829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:06.013141 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83c44c-7e68-40ae-b587-fd918b21c62d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:06.059030 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b285d3-1f5f-4359-bef3-a7a459f3bbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:06.059699 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b285d3-1f5f-4359-bef3-a7a459f3bbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:06.059986 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b285d3-1f5f-4359-bef3-a7a459f3bbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:06.060264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b285d3-1f5f-4359-bef3-a7a459f3bbe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:06.062417 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b285d3-1f5f-4359-bef3-a7a459f3bbe9 None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=10cc8d215a4045f1b83e4120a718db23, user.id=baf04811d48d4c3fa638546a709d7179) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:06.062773 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b285d3-1f5f-4359-bef3-a7a459f3bbe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:06.064263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b285d3-1f5f-4359-bef3-a7a459f3bbe9 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:06.080057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 47/94] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:05 2022] GET /identity/v3/role_assignments?user.id=baf04811d48d4c3fa638546a709d7179&scope.project.id=10cc8d215a4045f1b83e4120a718db23&role.id=5891920310b54787bd1152d736d2a3f1 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:06.080963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b285d3-1f5f-4359-bef3-a7a459f3bbe9 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:06.116088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:06.169711 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147a815-ecee-4422-bd7d-70b9067b95de None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:06.249072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cd73c-a35d-4284-ae87-0686f1990a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:06.249767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cd73c-a35d-4284-ae87-0686f1990a3d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:06.250060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cd73c-a35d-4284-ae87-0686f1990a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:06.250510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cd73c-a35d-4284-ae87-0686f1990a3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:06.251408 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cd73c-a35d-4284-ae87-0686f1990a3d None admin] RBAC: Adding query filter params (role.id=a82e24db1e3c4f868a586e122dae393e, scope.project.id=10cc8d215a4045f1b83e4120a718db23, user.id=b7bfc576d3154257a676f91304b0d7b4) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:06.251730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cd73c-a35d-4284-ae87-0686f1990a3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:06.253854 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cd73c-a35d-4284-ae87-0686f1990a3d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:06.269699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 48/95] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:05 2022] GET /identity/v3/role_assignments?user.id=b7bfc576d3154257a676f91304b0d7b4&scope.project.id=10cc8d215a4045f1b83e4120a718db23&role.id=a82e24db1e3c4f868a586e122dae393e => generated 259 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:06.307193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b285d3-1f5f-4359-bef3-a7a459f3bbe9 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:06.368091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382bd38-41f6-460b-9f6f-69ab7fd3bd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:06.368880 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382bd38-41f6-460b-9f6f-69ab7fd3bd7e None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:06.369166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382bd38-41f6-460b-9f6f-69ab7fd3bd7e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:06.369450 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382bd38-41f6-460b-9f6f-69ab7fd3bd7e None admin] PATH_INFO: `/v3/projects/21c5201b37ab4a83a73a8931758d7d1a` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:06.375911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382bd38-41f6-460b-9f6f-69ab7fd3bd7e None admin] RBAC: Authorizing `identity:get_project(project_id=21c5201b37ab4a83a73a8931758d7d1a)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:06.377441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382bd38-41f6-460b-9f6f-69ab7fd3bd7e None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:06.379702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 48/96] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:06 2022] GET /identity/v3/projects/21c5201b37ab4a83a73a8931758d7d1a => generated 312 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:06.386627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cd73c-a35d-4284-ae87-0686f1990a3d None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:06.538278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cd73c-a35d-4284-ae87-0686f1990a3d None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:06.582526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8dea09-8891-4212-a189-7a2d65e8abb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:06.583249 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8dea09-8891-4212-a189-7a2d65e8abb0 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:06.583629 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8dea09-8891-4212-a189-7a2d65e8abb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:06.583932 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8dea09-8891-4212-a189-7a2d65e8abb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:06.585038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8dea09-8891-4212-a189-7a2d65e8abb0 None admin] RBAC: Adding query filter params (role.id=3ab0ee91b3284a65a0dd5696a6250883, scope.project.id=21c5201b37ab4a83a73a8931758d7d1a, user.id=b7bfc576d3154257a676f91304b0d7b4) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:06.585479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8dea09-8891-4212-a189-7a2d65e8abb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:06.587097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8dea09-8891-4212-a189-7a2d65e8abb0 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:06.594879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 49/97] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:06 2022] GET /identity/v3/role_assignments?user.id=b7bfc576d3154257a676f91304b0d7b4&scope.project.id=21c5201b37ab4a83a73a8931758d7d1a&role.id=3ab0ee91b3284a65a0dd5696a6250883 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:07.759391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3bf46-90f5-4cfb-80f6-065a0ae7d6da None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:07.760170 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3bf46-90f5-4cfb-80f6-065a0ae7d6da None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:07.760170 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3bf46-90f5-4cfb-80f6-065a0ae7d6da None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:07.761345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 49/98] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:07.767616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3049ae1-94e9-43c2-9dc4-c9825cd28462 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:07.768181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3049ae1-94e9-43c2-9dc4-c9825cd28462 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:07.768181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3049ae1-94e9-43c2-9dc4-c9825cd28462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:07.794741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-f3049ae1-94e9-43c2-9dc4-c9825cd28462 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:07.796028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f3049ae1-94e9-43c2-9dc4-c9825cd28462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:07.798121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:07.880919 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 50/99] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:07.920139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3bf46-90f5-4cfb-80f6-065a0ae7d6da None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:07.961338 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:08.074491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3bf46-90f5-4cfb-80f6-065a0ae7d6da None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:08.125480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5fe4a-9a7a-4f2d-b14d-4c3dce5c0002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:08.126405 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5fe4a-9a7a-4f2d-b14d-4c3dce5c0002 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:08.126737 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5fe4a-9a7a-4f2d-b14d-4c3dce5c0002 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:08.127021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5fe4a-9a7a-4f2d-b14d-4c3dce5c0002 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:08.145465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5fe4a-9a7a-4f2d-b14d-4c3dce5c0002 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:08.153569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-2db5fe4a-9a7a-4f2d-b14d-4c3dce5c0002 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:08.153569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:08.153569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:08.153569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:08.153569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:08.153569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:08.153569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:08.153569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:08.153569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:08.153569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:08.153569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:08.153569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:08.153569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:08.153569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:08.153569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:08.153569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:08.154490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:08.154490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:08.154490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:08.154490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:08.154490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:08.154490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:08.154490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:08.154490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:08.154490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:08.154490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:08.154490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:08.154490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:08.154490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:08.154490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:08.154490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:08.154490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:08.155366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:08.155366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:08.155366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:08.155366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:08.155366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:08.155366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:08.155366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:08.155366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:08.155366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:08.155366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:08.155366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:08.155366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:08.155366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 50/100] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:08.163145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3049ae1-94e9-43c2-9dc4-c9825cd28462 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:08.337408 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3049ae1-94e9-43c2-9dc4-c9825cd28462 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:08.385852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c0cee-34ff-4cb9-8d62-c72dc5ce2d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:08.387016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c0cee-34ff-4cb9-8d62-c72dc5ce2d0e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:08.387524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c0cee-34ff-4cb9-8d62-c72dc5ce2d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:08.388020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c0cee-34ff-4cb9-8d62-c72dc5ce2d0e None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:08.388994 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c0cee-34ff-4cb9-8d62-c72dc5ce2d0e None admin] RBAC: Adding query filter params (name=member) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:08.390143 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c0cee-34ff-4cb9-8d62-c72dc5ce2d0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:08.391827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c0cee-34ff-4cb9-8d62-c72dc5ce2d0e None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:08.403128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:08.407352 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 51/101] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:08.413709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5fe4a-9a7a-4f2d-b14d-4c3dce5c0002 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:08.546239 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:08.571185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5fe4a-9a7a-4f2d-b14d-4c3dce5c0002 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:08.623515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963fe79-7b98-4a17-aaac-4358eec6c4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:08.624847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963fe79-7b98-4a17-aaac-4358eec6c4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:08.625139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963fe79-7b98-4a17-aaac-4358eec6c4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:08.625429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963fe79-7b98-4a17-aaac-4358eec6c4a4 None admin] PATH_INFO: `/v3/users/b7bfc576d3154257a676f91304b0d7b4` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:08.631322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963fe79-7b98-4a17-aaac-4358eec6c4a4 None admin] RBAC: Authorizing `identity:get_user(user_id=b7bfc576d3154257a676f91304b0d7b4)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:08.632957 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963fe79-7b98-4a17-aaac-4358eec6c4a4 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:08.634712 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 51/102] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:08 2022] GET /identity/v3/users/b7bfc576d3154257a676f91304b0d7b4 => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:08.641292 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c0cee-34ff-4cb9-8d62-c72dc5ce2d0e None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:08.794958 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c0cee-34ff-4cb9-8d62-c72dc5ce2d0e None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:08.811914 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6bae1-868d-4c0a-a5a9-693703305b4d None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:08.812955 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6bae1-868d-4c0a-a5a9-693703305b4d None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:08.813442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6bae1-868d-4c0a-a5a9-693703305b4d None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:08.815066 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 52/103] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:52:08.821895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73210a01-d1b6-45ce-bacc-72b8d20c738d None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:08.822661 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73210a01-d1b6-45ce-bacc-72b8d20c738d None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:08.823177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73210a01-d1b6-45ce-bacc-72b8d20c738d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:08.855824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-73210a01-d1b6-45ce-bacc-72b8d20c738d None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:08.857825 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-73210a01-d1b6-45ce-bacc-72b8d20c738d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:08.864179 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f5420-0647-4cdc-8889-6179e00bff4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:08.865396 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f5420-0647-4cdc-8889-6179e00bff4c None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:08.865885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f5420-0647-4cdc-8889-6179e00bff4c None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:08.866365 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f5420-0647-4cdc-8889-6179e00bff4c None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:08.872165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f5420-0647-4cdc-8889-6179e00bff4c None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:08.874494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f5420-0647-4cdc-8889-6179e00bff4c None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:08.876548 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 52/104] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:08 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:08.884500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f5420-0647-4cdc-8889-6179e00bff4c None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:08.983881 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 53/105] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:52:08.985762 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a08b8-f062-4f6f-aaa6-d979b96e13cb None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:08.986061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a08b8-f062-4f6f-aaa6-d979b96e13cb None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:08.986350 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a08b8-f062-4f6f-aaa6-d979b96e13cb None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:08.996777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 54/106] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:09.002885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:09.013884 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a08b8-f062-4f6f-aaa6-d979b96e13cb None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:09.067718 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f5420-0647-4cdc-8889-6179e00bff4c None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:09.119686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:09.120573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab36e8e-a569-4887-9059-8485903ba655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:09.121937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab36e8e-a569-4887-9059-8485903ba655 None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:09.122514 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab36e8e-a569-4887-9059-8485903ba655 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:09.122847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab36e8e-a569-4887-9059-8485903ba655 None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23/users/b7bfc576d3154257a676f91304b0d7b4/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:09.133800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab36e8e-a569-4887-9059-8485903ba655 None admin] RBAC: Authorizing `identity:create_grant(project_id=10cc8d215a4045f1b83e4120a718db23, user_id=b7bfc576d3154257a676f91304b0d7b4, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:09.133800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab36e8e-a569-4887-9059-8485903ba655 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:09.142551 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a08b8-f062-4f6f-aaa6-d979b96e13cb None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:09.179681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a8a4d-e31f-4349-a3c3-ae2a73f18fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:09.180495 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a8a4d-e31f-4349-a3c3-ae2a73f18fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:09.180784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a8a4d-e31f-4349-a3c3-ae2a73f18fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:09.181066 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a8a4d-e31f-4349-a3c3-ae2a73f18fa0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:09.192257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a8a4d-e31f-4349-a3c3-ae2a73f18fa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:09.200219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-309a8a4d-e31f-4349-a3c3-ae2a73f18fa0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:09.200219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:09.200219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:09.200219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:09.200219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:09.200219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:09.200219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:09.200219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:09.200219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:09.200219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:09.200219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:09.200219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:09.200219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:09.200219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:09.200219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:09.200219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:09.201390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:09.201390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:09.201390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:09.201390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:09.201390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:09.201390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:09.201390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:09.201390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:09.201390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:09.201390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:09.201390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:09.201390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:09.201390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:09.201390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:09.201390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:09.201390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:09.203461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:09.203461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:09.203461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:09.203461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:09.203461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:09.203461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:09.203461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:09.203461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:09.203461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:09.203461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:09.203461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:09.203461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:09.203461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 55/107] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:09.204579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116cd88-7b8b-46c1-a5f2-e6da72fedfbe None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:09.204579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116cd88-7b8b-46c1-a5f2-e6da72fedfbe None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:09.204579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116cd88-7b8b-46c1-a5f2-e6da72fedfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:09.224932 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-0116cd88-7b8b-46c1-a5f2-e6da72fedfbe None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:09.226225 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0116cd88-7b8b-46c1-a5f2-e6da72fedfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:09.245855 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 53/108] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:52:08 2022] PUT /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23/users/b7bfc576d3154257a676f91304b0d7b4/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:52:09.247988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763a6b6-046c-4bda-af34-12a340d2fe17 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:09.249637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763a6b6-046c-4bda-af34-12a340d2fe17 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:09.250234 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763a6b6-046c-4bda-af34-12a340d2fe17 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:09.252121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 54/109] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:09.254105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763a6b6-046c-4bda-af34-12a340d2fe17 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:09.353184 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 56/110] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:09.354654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f603c-04a7-49cb-b04a-412145990cf9 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:09.354962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f603c-04a7-49cb-b04a-412145990cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:09.355254 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f603c-04a7-49cb-b04a-412145990cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:09.399897 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-fd5f603c-04a7-49cb-b04a-412145990cf9 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:09.401075 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-fd5f603c-04a7-49cb-b04a-412145990cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:09.453751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763a6b6-046c-4bda-af34-12a340d2fe17 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:09.498046 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:09.509690 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb59c50-9f76-4559-a787-d12e3565ef19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:09.517564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb59c50-9f76-4559-a787-d12e3565ef19 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:09.517564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb59c50-9f76-4559-a787-d12e3565ef19 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:09.517564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb59c50-9f76-4559-a787-d12e3565ef19 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:09.517564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb59c50-9f76-4559-a787-d12e3565ef19 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:09.517564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb59c50-9f76-4559-a787-d12e3565ef19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:09.517564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb59c50-9f76-4559-a787-d12e3565ef19 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:09.528811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 55/111] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:09.531378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb59c50-9f76-4559-a787-d12e3565ef19 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:09.532069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:09.543728 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 57/112] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:09.545086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f603c-04a7-49cb-b04a-412145990cf9 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:09.669586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb59c50-9f76-4559-a787-d12e3565ef19 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:09.692703 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f603c-04a7-49cb-b04a-412145990cf9 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:09.738396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a4bc4-b8d3-4397-b841-d4a00cce68d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:09.739708 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a4bc4-b8d3-4397-b841-d4a00cce68d2 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:09.740285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a4bc4-b8d3-4397-b841-d4a00cce68d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:09.740754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a4bc4-b8d3-4397-b841-d4a00cce68d2 None admin] PATH_INFO: `/v3/users/124e6c742e5f441d862efe694b44d923` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:09.749061 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b80a4-73e5-4d89-92a8-76c084a3e8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:09.752162 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b80a4-73e5-4d89-92a8-76c084a3e8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:09.752162 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b80a4-73e5-4d89-92a8-76c084a3e8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:09.752162 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b80a4-73e5-4d89-92a8-76c084a3e8d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:09.754768 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b80a4-73e5-4d89-92a8-76c084a3e8d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:09.758691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b80a4-73e5-4d89-92a8-76c084a3e8d5 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:09.759068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a4bc4-b8d3-4397-b841-d4a00cce68d2 None admin] RBAC: Authorizing `identity:get_user(user_id=124e6c742e5f441d862efe694b44d923)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:09.760876 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a4bc4-b8d3-4397-b841-d4a00cce68d2 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:09.766565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 56/113] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:52:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:09.766565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b80a4-73e5-4d89-92a8-76c084a3e8d5 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:09.768944 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 58/114] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:09 2022] GET /identity/v3/users/124e6c742e5f441d862efe694b44d923 => generated 289 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:09.770321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfadb65-84f1-444a-92ac-e1dc94f9e9ee None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:09.770455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfadb65-84f1-444a-92ac-e1dc94f9e9ee None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:09.770703 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfadb65-84f1-444a-92ac-e1dc94f9e9ee None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:09.779810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 59/115] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:09 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:09.781930 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfadb65-84f1-444a-92ac-e1dc94f9e9ee None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:09.855843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:09.866887 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:09.913139 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b80a4-73e5-4d89-92a8-76c084a3e8d5 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:09.925064 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfadb65-84f1-444a-92ac-e1dc94f9e9ee None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:09.956239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e386f-9344-4465-a967-9d497d3eeb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:09.957052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e386f-9344-4465-a967-9d497d3eeb3f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:09.957346 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e386f-9344-4465-a967-9d497d3eeb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:09.957634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e386f-9344-4465-a967-9d497d3eeb3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:09.969165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e386f-9344-4465-a967-9d497d3eeb3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:09.970827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a18501-935a-4d8a-8542-5fab243aca47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:09.971712 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a18501-935a-4d8a-8542-5fab243aca47 None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:09.972218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a18501-935a-4d8a-8542-5fab243aca47 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:09.972715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a18501-935a-4d8a-8542-5fab243aca47 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:09.973620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a18501-935a-4d8a-8542-5fab243aca47 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:09.975305 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a18501-935a-4d8a-8542-5fab243aca47 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:09.977039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-ec3e386f-9344-4465-a967-9d497d3eeb3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:09.977039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:09.977039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:09.977039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:09.977039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:09.977039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:09.977039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:09.977039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:09.977039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:09.977039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:09.977039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:09.977039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:09.977039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:09.977039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:09.977039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:09.977039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:09.977992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:09.977992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:09.977992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:09.977992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:09.977992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:09.977992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:09.977992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:09.977992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:09.977992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:09.977992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:09.977992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:09.977992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:09.977992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:09.977992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:09.977992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:09.977992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:09.979009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:09.979009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:09.979009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:09.979009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:09.979009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:09.979009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:09.979009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:09.979009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:09.979009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:09.979009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:09.979009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:09.979009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:09.979009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 57/116] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:52:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:09.979740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e386f-9344-4465-a967-9d497d3eeb3f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:10.087186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 60/117] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Jul 18 14:52:09 2022] POST /identity/v3/projects => generated 302 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:52:10.087186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7077f03-3b59-4360-8d18-48db11417760 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:10.087186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7077f03-3b59-4360-8d18-48db11417760 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:10.087186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7077f03-3b59-4360-8d18-48db11417760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:10.091884 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e386f-9344-4465-a967-9d497d3eeb3f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:10.111994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-f7077f03-3b59-4360-8d18-48db11417760 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:10.113401 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-f7077f03-3b59-4360-8d18-48db11417760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:10.117908 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:10.143812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d488b-f543-4655-9dc9-28e280299389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:10.144552 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d488b-f543-4655-9dc9-28e280299389 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:10.144844 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d488b-f543-4655-9dc9-28e280299389 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:10.145126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d488b-f543-4655-9dc9-28e280299389 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:10.208931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 61/118] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:10.211043 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a77f45-b5f7-4939-9eed-8cf0c9878545 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:10.211604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a77f45-b5f7-4939-9eed-8cf0c9878545 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:10.212202 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a77f45-b5f7-4939-9eed-8cf0c9878545 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:10.218754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 62/119] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:10.220108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a77f45-b5f7-4939-9eed-8cf0c9878545 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:10.251046 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d488b-f543-4655-9dc9-28e280299389 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:10.252642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d488b-f543-4655-9dc9-28e280299389 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:10.315861 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:10.395891 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 58/120] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:10.399941 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e7da0-3390-4d16-9758-82fe36bca2c7 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:10.400233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e7da0-3390-4d16-9758-82fe36bca2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:10.400524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e7da0-3390-4d16-9758-82fe36bca2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:10.404360 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a77f45-b5f7-4939-9eed-8cf0c9878545 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:10.423953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-6d3e7da0-3390-4d16-9758-82fe36bca2c7 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:10.425519 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-6d3e7da0-3390-4d16-9758-82fe36bca2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:10.440759 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:10.448488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af53b6-0520-4fbf-ae85-e2e50814bffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:10.451455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af53b6-0520-4fbf-ae85-e2e50814bffd None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:10.451455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af53b6-0520-4fbf-ae85-e2e50814bffd None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:10.451455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af53b6-0520-4fbf-ae85-e2e50814bffd None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:10.451455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af53b6-0520-4fbf-ae85-e2e50814bffd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:10.451455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af53b6-0520-4fbf-ae85-e2e50814bffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:10.453054 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af53b6-0520-4fbf-ae85-e2e50814bffd None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:10.462890 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 63/121] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:10.463849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af53b6-0520-4fbf-ae85-e2e50814bffd None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:10.501340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 59/122] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:10.502307 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e7da0-3390-4d16-9758-82fe36bca2c7 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:10.580310 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af53b6-0520-4fbf-ae85-e2e50814bffd None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:10.581955 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:10.615706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e7da0-3390-4d16-9758-82fe36bca2c7 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:10.626856 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad0fb6-f340-4a35-a036-d0175a1146b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:10.627949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad0fb6-f340-4a35-a036-d0175a1146b4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:10.628400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad0fb6-f340-4a35-a036-d0175a1146b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:10.628866 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad0fb6-f340-4a35-a036-d0175a1146b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:10.641350 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad0fb6-f340-4a35-a036-d0175a1146b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:10.649702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-3dad0fb6-f340-4a35-a036-d0175a1146b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:10.649702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:10.649702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:10.649702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:10.649702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:10.649702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:10.649702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:10.649702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:10.649702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:10.649702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:10.649702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:10.649702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:10.649702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:10.649702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:10.649702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:10.649702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:10.650853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:10.650853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:10.650853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:10.650853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:10.650853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:10.650853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:10.650853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:10.650853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:10.650853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:10.650853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:10.650853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:10.650853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:10.650853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:10.650853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:10.650853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:10.650853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:10.651785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:10.651785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:10.651785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:10.651785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:10.651785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:10.651785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:10.651785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:10.651785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:10.651785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:10.651785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:10.651785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:10.651785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:10.654348 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 64/123] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:10.656354 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad0fb6-f340-4a35-a036-d0175a1146b4 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:10.657580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefbf1a-74ed-4e43-a74a-30edecea96cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:10.658511 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefbf1a-74ed-4e43-a74a-30edecea96cd None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:10.658835 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefbf1a-74ed-4e43-a74a-30edecea96cd None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:10.659120 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefbf1a-74ed-4e43-a74a-30edecea96cd None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:10.660445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fefbf1a-74ed-4e43-a74a-30edecea96cd None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:10.662997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fefbf1a-74ed-4e43-a74a-30edecea96cd None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:10.669419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 60/124] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:10 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:10.669419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefbf1a-74ed-4e43-a74a-30edecea96cd None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:10.727053 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:10.794706 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad0fb6-f340-4a35-a036-d0175a1146b4 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:10.840536 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefbf1a-74ed-4e43-a74a-30edecea96cd None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:10.847176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b708e3-589a-4988-941c-d87ce9d317b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:10.848568 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b708e3-589a-4988-941c-d87ce9d317b3 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:10.849014 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b708e3-589a-4988-941c-d87ce9d317b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:10.849462 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b708e3-589a-4988-941c-d87ce9d317b3 None admin] PATH_INFO: `/v3/users/baf04811d48d4c3fa638546a709d7179` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:10.851559 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b708e3-589a-4988-941c-d87ce9d317b3 None admin] RBAC: Authorizing `identity:get_user(user_id=baf04811d48d4c3fa638546a709d7179)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:10.854081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b708e3-589a-4988-941c-d87ce9d317b3 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:10.862900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 65/125] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:10 2022] GET /identity/v3/users/baf04811d48d4c3fa638546a709d7179 => generated 247 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:10.864419 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b708e3-589a-4988-941c-d87ce9d317b3 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:10.899588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76af6e8-8ea8-4df7-9f1b-aaeaa66dea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:10.901422 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76af6e8-8ea8-4df7-9f1b-aaeaa66dea4d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:10.901867 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76af6e8-8ea8-4df7-9f1b-aaeaa66dea4d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:10.902851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76af6e8-8ea8-4df7-9f1b-aaeaa66dea4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:10.919307 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76af6e8-8ea8-4df7-9f1b-aaeaa66dea4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:10.924712 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:10.936975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-b76af6e8-8ea8-4df7-9f1b-aaeaa66dea4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:10.936975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:10.936975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:10.936975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:10.936975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:10.936975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:10.936975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:10.936975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:10.936975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:10.936975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:10.936975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:10.936975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:10.936975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:10.936975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:10.936975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:10.936975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:10.938924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:10.938924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:10.938924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:10.938924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:10.938924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:10.938924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:10.938924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:10.938924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:10.938924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:10.938924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:10.938924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:10.938924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:10.938924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:10.938924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:10.938924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:10.938924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:10.939816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:10.939816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:10.939816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:10.939816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:10.939816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:10.939816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:10.939816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:10.939816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:10.939816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:10.939816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:10.939816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:10.939816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:10.942931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 61/126] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:52:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:10.944036 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76af6e8-8ea8-4df7-9f1b-aaeaa66dea4d None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:11.021228 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b708e3-589a-4988-941c-d87ce9d317b3 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:11.071115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae7bb5-4e26-4995-bad4-0a628c6a1815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:11.071984 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae7bb5-4e26-4995-bad4-0a628c6a1815 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:11.072385 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae7bb5-4e26-4995-bad4-0a628c6a1815 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:11.072780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae7bb5-4e26-4995-bad4-0a628c6a1815 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:11.073665 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae7bb5-4e26-4995-bad4-0a628c6a1815 None admin] RBAC: Adding query filter params (name=member) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:11.074082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae7bb5-4e26-4995-bad4-0a628c6a1815 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:11.075645 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae7bb5-4e26-4995-bad4-0a628c6a1815 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:11.077902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:11.089723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 66/127] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:11.090581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae7bb5-4e26-4995-bad4-0a628c6a1815 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:11.107056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76af6e8-8ea8-4df7-9f1b-aaeaa66dea4d None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:11.152899 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd5222-e434-4531-bfc7-6c08a131b207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:11.154157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd5222-e434-4531-bfc7-6c08a131b207 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:11.154904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd5222-e434-4531-bfc7-6c08a131b207 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:11.155340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd5222-e434-4531-bfc7-6c08a131b207 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:11.156490 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd5222-e434-4531-bfc7-6c08a131b207 None admin] RBAC: Adding query filter params (role.id=192149ed0eab4f039954e4617e7ee779, scope.project.id=10cc8d215a4045f1b83e4120a718db23, user.id=124e6c742e5f441d862efe694b44d923) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:11.156998 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd5222-e434-4531-bfc7-6c08a131b207 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:11.159126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd5222-e434-4531-bfc7-6c08a131b207 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:11.171661 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 62/128] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:10 2022] GET /identity/v3/role_assignments?user.id=124e6c742e5f441d862efe694b44d923&scope.project.id=10cc8d215a4045f1b83e4120a718db23&role.id=192149ed0eab4f039954e4617e7ee779 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:11.172779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd5222-e434-4531-bfc7-6c08a131b207 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:11.274905 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:11.289769 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae7bb5-4e26-4995-bad4-0a628c6a1815 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:11.361791 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2c6fc-1db0-43c6-a681-c912f715bbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:11.362731 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2c6fc-1db0-43c6-a681-c912f715bbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:11.363028 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2c6fc-1db0-43c6-a681-c912f715bbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:11.363320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2c6fc-1db0-43c6-a681-c912f715bbd8 None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:11.364607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb2c6fc-1db0-43c6-a681-c912f715bbd8 None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:11.366249 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb2c6fc-1db0-43c6-a681-c912f715bbd8 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:11.371448 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 67/129] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:11 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:11.371875 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2c6fc-1db0-43c6-a681-c912f715bbd8 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:11.406737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd5222-e434-4531-bfc7-6c08a131b207 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:11.496757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce6054-ab05-46b6-8d51-be8ef3ca0992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:11.497831 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce6054-ab05-46b6-8d51-be8ef3ca0992 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:11.498322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce6054-ab05-46b6-8d51-be8ef3ca0992 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:11.498912 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce6054-ab05-46b6-8d51-be8ef3ca0992 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:11.500075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce6054-ab05-46b6-8d51-be8ef3ca0992 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:11.500635 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce6054-ab05-46b6-8d51-be8ef3ca0992 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:11.502749 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce6054-ab05-46b6-8d51-be8ef3ca0992 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:11.522673 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 63/130] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:52:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:11.524035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce6054-ab05-46b6-8d51-be8ef3ca0992 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:11.530183 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:11.555011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2c6fc-1db0-43c6-a681-c912f715bbd8 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:11.618313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca990a0b-8b2e-443e-bee6-a1a497652125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:11.619261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca990a0b-8b2e-443e-bee6-a1a497652125 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:11.619563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca990a0b-8b2e-443e-bee6-a1a497652125 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:11.619845 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca990a0b-8b2e-443e-bee6-a1a497652125 None admin] PATH_INFO: `/v3/users/b7bfc576d3154257a676f91304b0d7b4` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:11.622202 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca990a0b-8b2e-443e-bee6-a1a497652125 None admin] RBAC: Authorizing `identity:get_user(user_id=b7bfc576d3154257a676f91304b0d7b4)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:11.624439 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca990a0b-8b2e-443e-bee6-a1a497652125 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:11.626156 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 68/131] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:11 2022] GET /identity/v3/users/b7bfc576d3154257a676f91304b0d7b4 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:11.626965 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca990a0b-8b2e-443e-bee6-a1a497652125 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:11.708758 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce6054-ab05-46b6-8d51-be8ef3ca0992 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:11.714922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:11.749554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197808b-10a1-41dc-9e4e-10843e2f4e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:11.750706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197808b-10a1-41dc-9e4e-10843e2f4e99 None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:11.751024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197808b-10a1-41dc-9e4e-10843e2f4e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:11.751315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197808b-10a1-41dc-9e4e-10843e2f4e99 None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23/users/baf04811d48d4c3fa638546a709d7179/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:11.753464 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197808b-10a1-41dc-9e4e-10843e2f4e99 None admin] RBAC: Authorizing `identity:create_grant(project_id=10cc8d215a4045f1b83e4120a718db23, user_id=baf04811d48d4c3fa638546a709d7179, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:11.755474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197808b-10a1-41dc-9e4e-10843e2f4e99 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:11.817221 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca990a0b-8b2e-443e-bee6-a1a497652125 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:11.858759 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 64/132] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:52:11 2022] PUT /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23/users/baf04811d48d4c3fa638546a709d7179/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:52:11.860650 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197808b-10a1-41dc-9e4e-10843e2f4e99 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:11.865409 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883759a4-b575-4d5b-9167-8752f505e7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:11.866426 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883759a4-b575-4d5b-9167-8752f505e7fa None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:11.866743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883759a4-b575-4d5b-9167-8752f505e7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:11.867024 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883759a4-b575-4d5b-9167-8752f505e7fa None admin] PATH_INFO: `/v3/users/b7bfc576d3154257a676f91304b0d7b4` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:11.871265 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883759a4-b575-4d5b-9167-8752f505e7fa None admin] RBAC: Authorizing `identity:get_user(user_id=b7bfc576d3154257a676f91304b0d7b4)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:11.875756 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883759a4-b575-4d5b-9167-8752f505e7fa None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:11.879391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 69/133] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:11 2022] GET /identity/v3/users/b7bfc576d3154257a676f91304b0d7b4 => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:11.883517 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4acb41-3787-4fd7-8646-1cbf54f66266 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:11.884210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4acb41-3787-4fd7-8646-1cbf54f66266 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:11.884210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4acb41-3787-4fd7-8646-1cbf54f66266 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:11.886867 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 70/134] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:11.888944 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4acb41-3787-4fd7-8646-1cbf54f66266 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:12.015452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:12.083411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197808b-10a1-41dc-9e4e-10843e2f4e99 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:12.090944 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:12.101217 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4acb41-3787-4fd7-8646-1cbf54f66266 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:12.128728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57419d3-e0bb-4555-8609-c49473fb4b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:12.130001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57419d3-e0bb-4555-8609-c49473fb4b30 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:12.130502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57419d3-e0bb-4555-8609-c49473fb4b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:12.131044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57419d3-e0bb-4555-8609-c49473fb4b30 None admin] PATH_INFO: `/v3/projects/21c5201b37ab4a83a73a8931758d7d1a` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:12.132661 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57419d3-e0bb-4555-8609-c49473fb4b30 None admin] RBAC: Authorizing `identity:get_project(project_id=21c5201b37ab4a83a73a8931758d7d1a)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:12.134377 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57419d3-e0bb-4555-8609-c49473fb4b30 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:12.136512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 65/135] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:11 2022] GET /identity/v3/projects/21c5201b37ab4a83a73a8931758d7d1a => generated 312 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:12.140140 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c620d-0ab3-435a-8d68-6ca40947726a None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:12.140714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c620d-0ab3-435a-8d68-6ca40947726a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:12.141218 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c620d-0ab3-435a-8d68-6ca40947726a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:12.142103 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acfdb1-027e-4171-8f8d-27af445d5ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:12.143038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acfdb1-027e-4171-8f8d-27af445d5ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:12.143326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acfdb1-027e-4171-8f8d-27af445d5ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:12.143615 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acfdb1-027e-4171-8f8d-27af445d5ab3 None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:12.145033 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74acfdb1-027e-4171-8f8d-27af445d5ab3 None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:12.146679 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74acfdb1-027e-4171-8f8d-27af445d5ab3 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:12.148763 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 71/136] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:11 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:12.149346 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acfdb1-027e-4171-8f8d-27af445d5ab3 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:12.164599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-0b8c620d-0ab3-435a-8d68-6ca40947726a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:12.166346 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-0b8c620d-0ab3-435a-8d68-6ca40947726a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:12.266924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acfdb1-027e-4171-8f8d-27af445d5ab3 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:12.280631 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 66/137] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:12.282266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c620d-0ab3-435a-8d68-6ca40947726a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:12.309709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42419781-bf80-4487-be23-3734cf06fbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:12.314355 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42419781-bf80-4487-be23-3734cf06fbd2 None admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:12.314355 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42419781-bf80-4487-be23-3734cf06fbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:12.314355 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42419781-bf80-4487-be23-3734cf06fbd2 None admin] PATH_INFO: `/v3/projects/21c5201b37ab4a83a73a8931758d7d1a/users/b7bfc576d3154257a676f91304b0d7b4/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:12.316122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42419781-bf80-4487-be23-3734cf06fbd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=21c5201b37ab4a83a73a8931758d7d1a, user_id=b7bfc576d3154257a676f91304b0d7b4, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:12.317610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42419781-bf80-4487-be23-3734cf06fbd2 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:12.376414 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:12.398175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 72/138] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:52:12 2022] PUT /identity/v3/projects/21c5201b37ab4a83a73a8931758d7d1a/users/b7bfc576d3154257a676f91304b0d7b4/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:52:12.399108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42419781-bf80-4487-be23-3734cf06fbd2 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:12.450490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:12.470317 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c620d-0ab3-435a-8d68-6ca40947726a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:12.541620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f0868-aab6-41d8-82fc-d77156f07b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:12.542572 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f0868-aab6-41d8-82fc-d77156f07b07 None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:12.542850 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f0868-aab6-41d8-82fc-d77156f07b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:12.543130 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f0868-aab6-41d8-82fc-d77156f07b07 None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23/users/b7bfc576d3154257a676f91304b0d7b4/roles/a82e24db1e3c4f868a586e122dae393e` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:12.545583 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f0868-aab6-41d8-82fc-d77156f07b07 None admin] RBAC: Authorizing `identity:create_grant(project_id=10cc8d215a4045f1b83e4120a718db23, user_id=b7bfc576d3154257a676f91304b0d7b4, role_id=a82e24db1e3c4f868a586e122dae393e)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:12.547132 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f0868-aab6-41d8-82fc-d77156f07b07 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:12.583530 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42419781-bf80-4487-be23-3734cf06fbd2 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:12.629045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66d809-d964-449f-a9b8-58ec9004cd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:12.630347 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66d809-d964-449f-a9b8-58ec9004cd11 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:12.630919 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66d809-d964-449f-a9b8-58ec9004cd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:12.631488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66d809-d964-449f-a9b8-58ec9004cd11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:12.646493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66d809-d964-449f-a9b8-58ec9004cd11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:12.656906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-8f66d809-d964-449f-a9b8-58ec9004cd11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:12.656906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:12.656906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:12.656906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:12.656906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:12.656906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:12.656906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:12.656906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:12.656906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:12.656906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:12.656906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:12.656906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:12.656906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:12.656906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:12.656906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:12.656906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:12.659525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:12.659525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:12.659525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:12.659525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:12.659525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:12.659525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:12.659525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:12.659525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:12.659525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:12.659525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:12.659525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:12.659525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:12.659525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:12.659525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:12.659525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:12.659525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:12.660311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:12.660311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:12.660311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:12.660311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:12.660311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:12.660311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:12.660311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:12.660311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:12.660311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:12.660311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:12.660311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:12.660311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:12.660903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 67/139] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:52:12 2022] PUT /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23/users/b7bfc576d3154257a676f91304b0d7b4/roles/a82e24db1e3c4f868a586e122dae393e => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:52:12.660903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b2b03-5d7b-4247-a6b8-57c96ae3b585 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:12.660903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b2b03-5d7b-4247-a6b8-57c96ae3b585 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:12.661086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 73/140] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:12.661152 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b2b03-5d7b-4247-a6b8-57c96ae3b585 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:12.661152 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 68/141] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:12.668388 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b712179-410a-41b0-8e76-5204a7157690 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:12.668388 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b712179-410a-41b0-8e76-5204a7157690 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:12.668652 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b2b03-5d7b-4247-a6b8-57c96ae3b585 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:12.668724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b712179-410a-41b0-8e76-5204a7157690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:12.697168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-3b712179-410a-41b0-8e76-5204a7157690 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:12.698407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-3b712179-410a-41b0-8e76-5204a7157690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:12.804509 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b2b03-5d7b-4247-a6b8-57c96ae3b585 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:12.810713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 74/142] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:12.827344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:12.843435 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaffdf6-cdaa-46a0-9c5e-04a24fa28e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:12.844131 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaffdf6-cdaa-46a0-9c5e-04a24fa28e1d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:12.844420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaffdf6-cdaa-46a0-9c5e-04a24fa28e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:12.844701 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaffdf6-cdaa-46a0-9c5e-04a24fa28e1d None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:12.845467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaffdf6-cdaa-46a0-9c5e-04a24fa28e1d None admin] RBAC: Adding query filter params (name=member) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:12.845775 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaffdf6-cdaa-46a0-9c5e-04a24fa28e1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:12.847311 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaffdf6-cdaa-46a0-9c5e-04a24fa28e1d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:12.856985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 69/143] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:12.857397 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b712179-410a-41b0-8e76-5204a7157690 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:12.866825 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaffdf6-cdaa-46a0-9c5e-04a24fa28e1d None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:12.914507 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:13.090817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaffdf6-cdaa-46a0-9c5e-04a24fa28e1d None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:13.104728 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b712179-410a-41b0-8e76-5204a7157690 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:13.155880 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b836e49-e116-4144-9677-0117cc8e538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:13.156848 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b836e49-e116-4144-9677-0117cc8e538f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:13.157133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b836e49-e116-4144-9677-0117cc8e538f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:13.157408 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b836e49-e116-4144-9677-0117cc8e538f None admin] PATH_INFO: `/v3/users/b7bfc576d3154257a676f91304b0d7b4` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:13.158755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b836e49-e116-4144-9677-0117cc8e538f None admin] RBAC: Authorizing `identity:get_user(user_id=b7bfc576d3154257a676f91304b0d7b4)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:13.160250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b836e49-e116-4144-9677-0117cc8e538f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:13.161932 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 70/144] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:12 2022] GET /identity/v3/users/b7bfc576d3154257a676f91304b0d7b4 => generated 275 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:13.164287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea45a55-7419-4efc-baca-633ec08ed0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:13.165170 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea45a55-7419-4efc-baca-633ec08ed0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:13.165410 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea45a55-7419-4efc-baca-633ec08ed0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:13.165696 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea45a55-7419-4efc-baca-633ec08ed0c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:13.167669 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b836e49-e116-4144-9677-0117cc8e538f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:13.171383 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea45a55-7419-4efc-baca-633ec08ed0c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:13.172909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea45a55-7419-4efc-baca-633ec08ed0c2 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:13.174909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 75/145] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:52:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:13.180758 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea45a55-7419-4efc-baca-633ec08ed0c2 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:13.274251 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:13.290994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:13.311944 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea45a55-7419-4efc-baca-633ec08ed0c2 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:13.352086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019aab1-eb69-4226-b3bd-a182ac1e5caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:13.352779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019aab1-eb69-4226-b3bd-a182ac1e5caf None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:13.358565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019aab1-eb69-4226-b3bd-a182ac1e5caf None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:13.358565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019aab1-eb69-4226-b3bd-a182ac1e5caf None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:13.358565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019aab1-eb69-4226-b3bd-a182ac1e5caf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:13.358565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019aab1-eb69-4226-b3bd-a182ac1e5caf None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:13.419401 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b836e49-e116-4144-9677-0117cc8e538f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:13.477591 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 76/146] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:52:13 2022] POST /identity/v3/users => generated 283 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:52:13.484848 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147b27a-3323-4de5-9039-c6c8cfa1a44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:13.486211 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147b27a-3323-4de5-9039-c6c8cfa1a44e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:13.486792 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147b27a-3323-4de5-9039-c6c8cfa1a44e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:13.487302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147b27a-3323-4de5-9039-c6c8cfa1a44e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:13.596023 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147b27a-3323-4de5-9039-c6c8cfa1a44e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:13.597563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147b27a-3323-4de5-9039-c6c8cfa1a44e None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:13.664861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:13.735304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 71/147] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:13.742850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019aab1-eb69-4226-b3bd-a182ac1e5caf None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:13.876326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019aab1-eb69-4226-b3bd-a182ac1e5caf None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:13.920915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d970d7e-a9d2-4cde-ae29-02a113aba509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:13.927176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d970d7e-a9d2-4cde-ae29-02a113aba509 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:13.927481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d970d7e-a9d2-4cde-ae29-02a113aba509 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:13.927765 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d970d7e-a9d2-4cde-ae29-02a113aba509 None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:13.929317 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d970d7e-a9d2-4cde-ae29-02a113aba509 None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:13.930906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d970d7e-a9d2-4cde-ae29-02a113aba509 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:13.932872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 77/148] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:13 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:13.940080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147b27a-3323-4de5-9039-c6c8cfa1a44e None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:14.070100 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147b27a-3323-4de5-9039-c6c8cfa1a44e None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:14.114594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0f907-2eb7-4a8c-a0bb-9c5d4453a035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:14.115364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0f907-2eb7-4a8c-a0bb-9c5d4453a035 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:14.115598 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0f907-2eb7-4a8c-a0bb-9c5d4453a035 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:14.116000 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0f907-2eb7-4a8c-a0bb-9c5d4453a035 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:14.116965 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0f907-2eb7-4a8c-a0bb-9c5d4453a035 None admin] RBAC: Adding query filter params (role.id=3ab0ee91b3284a65a0dd5696a6250883, scope.project.id=10cc8d215a4045f1b83e4120a718db23, user.id=b7bfc576d3154257a676f91304b0d7b4) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:14.117309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0f907-2eb7-4a8c-a0bb-9c5d4453a035 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:14.118808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0f907-2eb7-4a8c-a0bb-9c5d4453a035 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:14.126372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 72/149] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:13 2022] GET /identity/v3/role_assignments?user.id=b7bfc576d3154257a676f91304b0d7b4&scope.project.id=10cc8d215a4045f1b83e4120a718db23&role.id=3ab0ee91b3284a65a0dd5696a6250883 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:14.143802 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de944d8-280a-46d1-9f2b-0d024408cd1b None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:14.144973 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de944d8-280a-46d1-9f2b-0d024408cd1b None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:14.145444 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de944d8-280a-46d1-9f2b-0d024408cd1b None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:14.148886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 78/150] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:14.155687 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bdec1-c728-4951-97db-c3696b2d051a None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:14.156047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bdec1-c728-4951-97db-c3696b2d051a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:14.156282 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bdec1-c728-4951-97db-c3696b2d051a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:14.183934 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-c50bdec1-c728-4951-97db-c3696b2d051a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:14.185329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c50bdec1-c728-4951-97db-c3696b2d051a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:14.196738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:14.277673 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 73/151] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:52:14.313082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de944d8-280a-46d1-9f2b-0d024408cd1b None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:14.330691 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:14.467092 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de944d8-280a-46d1-9f2b-0d024408cd1b None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:14.518158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341eb4f-2cb0-4fd8-8e01-d3c31e80a2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:14.519396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341eb4f-2cb0-4fd8-8e01-d3c31e80a2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:14.519925 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341eb4f-2cb0-4fd8-8e01-d3c31e80a2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:14.520500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341eb4f-2cb0-4fd8-8e01-d3c31e80a2a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:14.539391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341eb4f-2cb0-4fd8-8e01-d3c31e80a2a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:14.550722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-9341eb4f-2cb0-4fd8-8e01-d3c31e80a2a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:14.550722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:14.550722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:14.550722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:14.550722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:14.550722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:14.550722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:14.550722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:14.550722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:14.550722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:14.550722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:14.550722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:14.550722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:14.550722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:14.550722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:14.550722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:14.551810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:14.551810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:14.551810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:14.551810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:14.551810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:14.551810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:14.551810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:14.551810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:14.551810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:14.551810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:14.551810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:14.551810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:14.551810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:14.551810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:14.551810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:14.551810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:14.552694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:14.552694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:14.552694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:14.552694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:14.552694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:14.552694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:14.552694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:14.552694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:14.552694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:14.552694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:14.552694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:14.552694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:14.555330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 79/152] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:14.563433 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50bdec1-c728-4951-97db-c3696b2d051a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:14.702866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50bdec1-c728-4951-97db-c3696b2d051a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:14.724897 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b897a1-9a3e-4b80-8a63-f72d7573ff7a None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:14.725562 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b897a1-9a3e-4b80-8a63-f72d7573ff7a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:14.726051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b897a1-9a3e-4b80-8a63-f72d7573ff7a None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:14.728225 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 80/153] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:52:14.735933 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e22e32-8306-418a-98b5-b11c58e52b8b None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:14.736751 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e22e32-8306-418a-98b5-b11c58e52b8b None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:14.737251 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e22e32-8306-418a-98b5-b11c58e52b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:14.769206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e8ce2-9075-481f-bd33-f31e6b72f0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:14.769918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e8ce2-9075-481f-bd33-f31e6b72f0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:14.770414 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e8ce2-9075-481f-bd33-f31e6b72f0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:14.770951 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e8ce2-9075-481f-bd33-f31e6b72f0d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:14.772168 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e8ce2-9075-481f-bd33-f31e6b72f0d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:14.774722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e8ce2-9075-481f-bd33-f31e6b72f0d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:14.774722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e8ce2-9075-481f-bd33-f31e6b72f0d2 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:14.776497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-a9e22e32-8306-418a-98b5-b11c58e52b8b None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:14.778307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a9e22e32-8306-418a-98b5-b11c58e52b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:14.793000 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 74/154] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:14.800482 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e8ce2-9075-481f-bd33-f31e6b72f0d2 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:14.834959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:14.964013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:14.970740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 81/155] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:52:15.007620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e22e32-8306-418a-98b5-b11c58e52b8b None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:15.052685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e8ce2-9075-481f-bd33-f31e6b72f0d2 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:15.165315 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e22e32-8306-418a-98b5-b11c58e52b8b None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:15.166075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaacb60-55a4-40fc-84b8-1e070459a27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:15.166901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaacb60-55a4-40fc-84b8-1e070459a27a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:15.167111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaacb60-55a4-40fc-84b8-1e070459a27a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:15.167504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaacb60-55a4-40fc-84b8-1e070459a27a None admin] PATH_INFO: `/v3/users/124e6c742e5f441d862efe694b44d923` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:15.169895 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaacb60-55a4-40fc-84b8-1e070459a27a None admin] RBAC: Authorizing `identity:get_user(user_id=124e6c742e5f441d862efe694b44d923)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:15.171995 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaacb60-55a4-40fc-84b8-1e070459a27a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:15.173792 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 75/156] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:14 2022] GET /identity/v3/users/124e6c742e5f441d862efe694b44d923 => generated 289 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:15.189699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaacb60-55a4-40fc-84b8-1e070459a27a None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:15.212847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d100d-2535-4a37-91ae-50f95d38524a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:15.213493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d100d-2535-4a37-91ae-50f95d38524a None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:15.213794 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d100d-2535-4a37-91ae-50f95d38524a None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:15.214210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d100d-2535-4a37-91ae-50f95d38524a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:15.230175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d100d-2535-4a37-91ae-50f95d38524a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:15.252067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-7d1d100d-2535-4a37-91ae-50f95d38524a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:15.252067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:15.252067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:15.252067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:15.252067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:15.252067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:15.252067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:15.252067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:15.252067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:15.252067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:15.252067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:15.252067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:15.252067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:15.252067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:15.252067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:15.252067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:15.254369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:15.254369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:15.254369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:15.254369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:15.254369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:15.254369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:15.254369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:15.254369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:15.254369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:15.254369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:15.254369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:15.254369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:15.254369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:15.254369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:15.254369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:15.254369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:15.259178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:15.259178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:15.259178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:15.259178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:15.259178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:15.259178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:15.259178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:15.259178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:15.259178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:15.259178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:15.259178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:15.259178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:15.259178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 82/157] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:52:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:15.267507 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d100d-2535-4a37-91ae-50f95d38524a None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:15.323033 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:15.374407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaacb60-55a4-40fc-84b8-1e070459a27a None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:15.375134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:15.460954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d100d-2535-4a37-91ae-50f95d38524a None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:15.463990 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65030e12-cdc5-4d4a-8c31-a61d7ea9ca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:15.464986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65030e12-cdc5-4d4a-8c31-a61d7ea9ca02 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:15.465211 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65030e12-cdc5-4d4a-8c31-a61d7ea9ca02 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:15.465474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65030e12-cdc5-4d4a-8c31-a61d7ea9ca02 None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:15.471494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65030e12-cdc5-4d4a-8c31-a61d7ea9ca02 None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:15.473034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65030e12-cdc5-4d4a-8c31-a61d7ea9ca02 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:15.474948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 76/158] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:15 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:15.483780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65030e12-cdc5-4d4a-8c31-a61d7ea9ca02 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:15.548280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebb87b-b1a6-4f44-988c-987e3d6c8056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:15.549340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebb87b-b1a6-4f44-988c-987e3d6c8056 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:15.549820 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebb87b-b1a6-4f44-988c-987e3d6c8056 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:15.550287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebb87b-b1a6-4f44-988c-987e3d6c8056 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:15.551273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebb87b-b1a6-4f44-988c-987e3d6c8056 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:15.551977 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebb87b-b1a6-4f44-988c-987e3d6c8056 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:15.558566 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebb87b-b1a6-4f44-988c-987e3d6c8056 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:15.578967 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 83/159] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:15.586031 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebb87b-b1a6-4f44-988c-987e3d6c8056 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:15.646716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65030e12-cdc5-4d4a-8c31-a61d7ea9ca02 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:15.687063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e01a3-8234-4170-8d71-25d79f53cdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:15.687063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e01a3-8234-4170-8d71-25d79f53cdf2 None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:15.687063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e01a3-8234-4170-8d71-25d79f53cdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:15.687063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e01a3-8234-4170-8d71-25d79f53cdf2 None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23/users/124e6c742e5f441d862efe694b44d923/roles/192149ed0eab4f039954e4617e7ee779` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:15.689231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e01a3-8234-4170-8d71-25d79f53cdf2 None admin] RBAC: Authorizing `identity:create_grant(project_id=10cc8d215a4045f1b83e4120a718db23, user_id=124e6c742e5f441d862efe694b44d923, role_id=192149ed0eab4f039954e4617e7ee779)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:15.690832 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e01a3-8234-4170-8d71-25d79f53cdf2 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:15.774488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 77/160] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:52:15 2022] PUT /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23/users/124e6c742e5f441d862efe694b44d923/roles/192149ed0eab4f039954e4617e7ee779 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:52:15.776717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebb87b-b1a6-4f44-988c-987e3d6c8056 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:15.804410 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:15.827781 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547186b-5aad-4b84-bc6c-00cfc2cf236b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:15.828746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547186b-5aad-4b84-bc6c-00cfc2cf236b None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:15.829045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547186b-5aad-4b84-bc6c-00cfc2cf236b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:15.829321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547186b-5aad-4b84-bc6c-00cfc2cf236b None admin] PATH_INFO: `/v3/users/baf04811d48d4c3fa638546a709d7179` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:15.831314 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547186b-5aad-4b84-bc6c-00cfc2cf236b None admin] RBAC: Authorizing `identity:get_user(user_id=baf04811d48d4c3fa638546a709d7179)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:15.833175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280011d-d2c8-44de-aef9-14eb35a88917 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:15.833467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280011d-d2c8-44de-aef9-14eb35a88917 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:15.833749 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280011d-d2c8-44de-aef9-14eb35a88917 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:15.835608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 78/161] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:15.837254 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547186b-5aad-4b84-bc6c-00cfc2cf236b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:15.839207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 84/162] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:15 2022] GET /identity/v3/users/baf04811d48d4c3fa638546a709d7179 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:15.845464 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280011d-d2c8-44de-aef9-14eb35a88917 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:15.847784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc549f65-2935-41b2-8430-7edbd9835db4 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:15.848080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc549f65-2935-41b2-8430-7edbd9835db4 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:15.848367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc549f65-2935-41b2-8430-7edbd9835db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:15.848797 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:15.884483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-dc549f65-2935-41b2-8430-7edbd9835db4 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:15.885828 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-dc549f65-2935-41b2-8430-7edbd9835db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:16.077378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 85/163] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:16.116507 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280011d-d2c8-44de-aef9-14eb35a88917 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:16.126332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc549f65-2935-41b2-8430-7edbd9835db4 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:16.175411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541e574-d67c-40e9-9989-278cc2451efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:16.176571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541e574-d67c-40e9-9989-278cc2451efd None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:16.176864 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541e574-d67c-40e9-9989-278cc2451efd None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:16.177150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541e574-d67c-40e9-9989-278cc2451efd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:16.314770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541e574-d67c-40e9-9989-278cc2451efd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:16.316466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541e574-d67c-40e9-9989-278cc2451efd None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:16.318722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:16.364905 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc549f65-2935-41b2-8430-7edbd9835db4 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:16.406914 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:16.432000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f471d2b-d097-458a-a346-6f170d5bea7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:16.432825 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f471d2b-d097-458a-a346-6f170d5bea7c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:16.433108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f471d2b-d097-458a-a346-6f170d5bea7c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:16.433446 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f471d2b-d097-458a-a346-6f170d5bea7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:16.460289 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f471d2b-d097-458a-a346-6f170d5bea7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:16.476490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-7f471d2b-d097-458a-a346-6f170d5bea7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:16.476490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:16.476490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:16.476490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:16.476490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:16.476490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:16.476490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:16.476490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:16.476490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:16.476490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:16.476490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:16.476490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:16.476490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:16.476490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:16.476490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:16.476490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:16.477479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:16.477479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:16.477479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:16.477479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:16.477479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:16.477479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:16.477479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:16.477479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:16.477479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:16.477479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:16.477479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:16.477479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:16.477479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:16.477479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:16.477479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:16.477479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:16.478274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:16.478274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:16.478274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:16.478274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:16.478274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:16.478274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:16.478274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:16.478274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:16.478274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:16.478274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:16.478274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:16.478274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:16.478274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 86/164] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:16.484498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f471d2b-d097-458a-a346-6f170d5bea7c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:16.531765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 79/165] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:16.552458 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541e574-d67c-40e9-9989-278cc2451efd None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:16.611459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f471d2b-d097-458a-a346-6f170d5bea7c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:16.659185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566af73-9e21-46e1-ae27-224b15899992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:16.659977 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566af73-9e21-46e1-ae27-224b15899992 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:16.660265 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566af73-9e21-46e1-ae27-224b15899992 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:16.660547 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566af73-9e21-46e1-ae27-224b15899992 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:16.661322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566af73-9e21-46e1-ae27-224b15899992 None admin] RBAC: Adding query filter params (name=member) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:16.661634 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566af73-9e21-46e1-ae27-224b15899992 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:16.663085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566af73-9e21-46e1-ae27-224b15899992 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:16.684897 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 87/166] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:16.686927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb9a86-fc0b-4ad6-bb3c-a04694ed49d4 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:16.687421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb9a86-fc0b-4ad6-bb3c-a04694ed49d4 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:16.687714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb9a86-fc0b-4ad6-bb3c-a04694ed49d4 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:16.689114 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 88/167] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:16.699603 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb9a86-fc0b-4ad6-bb3c-a04694ed49d4 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:16.754300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541e574-d67c-40e9-9989-278cc2451efd None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:16.769111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:16.823928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498eee38-d6c8-4c52-bbe1-5c0d62961461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:16.824827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498eee38-d6c8-4c52-bbe1-5c0d62961461 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:16.825111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498eee38-d6c8-4c52-bbe1-5c0d62961461 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:16.825390 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498eee38-d6c8-4c52-bbe1-5c0d62961461 None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:16.827395 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498eee38-d6c8-4c52-bbe1-5c0d62961461 None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:16.828894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498eee38-d6c8-4c52-bbe1-5c0d62961461 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:16.830878 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 80/168] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:16 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:16.832784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8460be-093d-4698-8289-0e059cf0c322 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:16.833030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8460be-093d-4698-8289-0e059cf0c322 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:16.833313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8460be-093d-4698-8289-0e059cf0c322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:16.834944 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:16.860285 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-6e8460be-093d-4698-8289-0e059cf0c322 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:16.861542 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-6e8460be-093d-4698-8289-0e059cf0c322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:16.985228 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 81/169] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:16.990707 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8460be-093d-4698-8289-0e059cf0c322 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:17.001958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb9a86-fc0b-4ad6-bb3c-a04694ed49d4 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:17.062526 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de8636-6665-44f4-90f2-4b19bc463977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:17.063522 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de8636-6665-44f4-90f2-4b19bc463977 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:17.063810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de8636-6665-44f4-90f2-4b19bc463977 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:17.064088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de8636-6665-44f4-90f2-4b19bc463977 None admin] PATH_INFO: `/v3/users/b7bfc576d3154257a676f91304b0d7b4` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:17.065418 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de8636-6665-44f4-90f2-4b19bc463977 None admin] RBAC: Authorizing `identity:get_user(user_id=b7bfc576d3154257a676f91304b0d7b4)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:17.066966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de8636-6665-44f4-90f2-4b19bc463977 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:17.069543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 89/170] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:16 2022] GET /identity/v3/users/b7bfc576d3154257a676f91304b0d7b4 => generated 275 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:17.070773 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de8636-6665-44f4-90f2-4b19bc463977 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:17.185923 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:17.189578 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8460be-093d-4698-8289-0e059cf0c322 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:17.219911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de8636-6665-44f4-90f2-4b19bc463977 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:17.228767 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:17.272867 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3caf23-c444-49df-bc60-2ca7ea9b559b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:17.273547 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3caf23-c444-49df-bc60-2ca7ea9b559b None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:17.273835 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3caf23-c444-49df-bc60-2ca7ea9b559b None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:17.274116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3caf23-c444-49df-bc60-2ca7ea9b559b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:17.275116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3caf23-c444-49df-bc60-2ca7ea9b559b None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=10cc8d215a4045f1b83e4120a718db23, user.id=baf04811d48d4c3fa638546a709d7179) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:17.275440 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3caf23-c444-49df-bc60-2ca7ea9b559b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:17.276933 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3caf23-c444-49df-bc60-2ca7ea9b559b None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:17.278684 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad93829-fbfa-4940-8a90-f849dde7b19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:17.279826 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad93829-fbfa-4940-8a90-f849dde7b19c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:17.280504 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad93829-fbfa-4940-8a90-f849dde7b19c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:17.280963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad93829-fbfa-4940-8a90-f849dde7b19c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:17.291487 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 82/171] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:16 2022] GET /identity/v3/role_assignments?user.id=baf04811d48d4c3fa638546a709d7179&scope.project.id=10cc8d215a4045f1b83e4120a718db23&role.id=5891920310b54787bd1152d736d2a3f1 => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:17.292374 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3caf23-c444-49df-bc60-2ca7ea9b559b None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:17.295538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad93829-fbfa-4940-8a90-f849dde7b19c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:17.306271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-2ad93829-fbfa-4940-8a90-f849dde7b19c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:17.306271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:17.306271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:17.306271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:17.306271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:17.306271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:17.306271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:17.306271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:17.306271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:17.306271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:17.306271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:17.306271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:17.306271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:17.306271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:17.306271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:17.306271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:17.307319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:17.307319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:17.307319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:17.307319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:17.307319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:17.307319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:17.307319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:17.307319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:17.307319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:17.307319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:17.307319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:17.307319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:17.307319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:17.307319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:17.307319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:17.307319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:17.308208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:17.308208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:17.308208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:17.308208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:17.308208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:17.308208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:17.308208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:17.308208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:17.308208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:17.308208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:17.308208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:17.308208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:17.311470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 90/172] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:52:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:17.324407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad93829-fbfa-4940-8a90-f849dde7b19c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:17.517744 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3caf23-c444-49df-bc60-2ca7ea9b559b None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:17.522789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad93829-fbfa-4940-8a90-f849dde7b19c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:17.592471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55240a28-8a6a-4556-8d24-4cd9008dca32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:17.593158 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55240a28-8a6a-4556-8d24-4cd9008dca32 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:17.593459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55240a28-8a6a-4556-8d24-4cd9008dca32 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:17.593735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55240a28-8a6a-4556-8d24-4cd9008dca32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:17.611326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97b197-5cff-4194-b709-494205059828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:17.612052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97b197-5cff-4194-b709-494205059828 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:17.612349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97b197-5cff-4194-b709-494205059828 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:17.612633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97b197-5cff-4194-b709-494205059828 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:17.613425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97b197-5cff-4194-b709-494205059828 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:17.613738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97b197-5cff-4194-b709-494205059828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:17.615213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97b197-5cff-4194-b709-494205059828 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:17.642915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 91/173] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:52:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:17.648732 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97b197-5cff-4194-b709-494205059828 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:17.664373 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:17.719836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:17.742982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55240a28-8a6a-4556-8d24-4cd9008dca32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:17.744535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55240a28-8a6a-4556-8d24-4cd9008dca32 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:17.902776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97b197-5cff-4194-b709-494205059828 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:17.971998 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0fd14-bdfe-4ff0-9313-b361b9beb152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:17.972893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0fd14-bdfe-4ff0-9313-b361b9beb152 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:17.973184 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0fd14-bdfe-4ff0-9313-b361b9beb152 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:17.973467 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0fd14-bdfe-4ff0-9313-b361b9beb152 None admin] PATH_INFO: `/v3/users/b7bfc576d3154257a676f91304b0d7b4` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:17.974744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0fd14-bdfe-4ff0-9313-b361b9beb152 None admin] RBAC: Authorizing `identity:get_user(user_id=b7bfc576d3154257a676f91304b0d7b4)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:17.976606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0fd14-bdfe-4ff0-9313-b361b9beb152 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:17.978406 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 92/174] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:17 2022] GET /identity/v3/users/b7bfc576d3154257a676f91304b0d7b4 => generated 275 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:17.986221 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0fd14-bdfe-4ff0-9313-b361b9beb152 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:18.033853 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 83/175] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 742 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:18.040421 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55240a28-8a6a-4556-8d24-4cd9008dca32 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:18.159726 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:18.170894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0fd14-bdfe-4ff0-9313-b361b9beb152 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:18.195641 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:18.236546 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079decdb-db2c-451f-a276-7fdf7dd7d44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:18.237495 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079decdb-db2c-451f-a276-7fdf7dd7d44d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:18.237941 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079decdb-db2c-451f-a276-7fdf7dd7d44d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:18.238381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079decdb-db2c-451f-a276-7fdf7dd7d44d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:18.240518 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55240a28-8a6a-4556-8d24-4cd9008dca32 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:18.303107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092f9b0-0f81-45d1-ac23-1c9b02f91fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:18.304335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092f9b0-0f81-45d1-ac23-1c9b02f91fbc None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:18.304779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092f9b0-0f81-45d1-ac23-1c9b02f91fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:18.305209 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092f9b0-0f81-45d1-ac23-1c9b02f91fbc None admin] PATH_INFO: `/v3/projects/21c5201b37ab4a83a73a8931758d7d1a` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:18.306625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092f9b0-0f81-45d1-ac23-1c9b02f91fbc None admin] RBAC: Authorizing `identity:get_project(project_id=21c5201b37ab4a83a73a8931758d7d1a)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:18.308290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092f9b0-0f81-45d1-ac23-1c9b02f91fbc None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:18.311230 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 84/176] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:18 2022] GET /identity/v3/projects/21c5201b37ab4a83a73a8931758d7d1a => generated 312 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:18.312675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ceef9-118c-490e-94a3-15b2e63ac822 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:18.313123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ceef9-118c-490e-94a3-15b2e63ac822 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:18.313557 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ceef9-118c-490e-94a3-15b2e63ac822 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:18.315062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 85/177] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:18.329340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e864f5-dabf-4537-a4a4-9aa0f25e6905 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:18.331253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e864f5-dabf-4537-a4a4-9aa0f25e6905 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:18.331701 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e864f5-dabf-4537-a4a4-9aa0f25e6905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:18.355545 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-f3e864f5-dabf-4537-a4a4-9aa0f25e6905 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:18.357982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f3e864f5-dabf-4537-a4a4-9aa0f25e6905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:18.379722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079decdb-db2c-451f-a276-7fdf7dd7d44d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:18.381633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079decdb-db2c-451f-a276-7fdf7dd7d44d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:18.444235 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 86/178] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:18.445522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e864f5-dabf-4537-a4a4-9aa0f25e6905 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:18.528421 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:18.570168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:18.575781 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 93/179] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 14:52:18.576948 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079decdb-db2c-451f-a276-7fdf7dd7d44d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:18.585160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e864f5-dabf-4537-a4a4-9aa0f25e6905 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:18.643024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be8827-5151-4462-b08d-95094abce09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:18.645952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be8827-5151-4462-b08d-95094abce09f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:18.646680 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be8827-5151-4462-b08d-95094abce09f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:18.647267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be8827-5151-4462-b08d-95094abce09f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:18.648645 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be8827-5151-4462-b08d-95094abce09f None admin] RBAC: Adding query filter params (role.id=3ab0ee91b3284a65a0dd5696a6250883, scope.project.id=21c5201b37ab4a83a73a8931758d7d1a, user.id=b7bfc576d3154257a676f91304b0d7b4) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:18.649193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be8827-5151-4462-b08d-95094abce09f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:18.651415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be8827-5151-4462-b08d-95094abce09f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:18.661517 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 87/180] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:18 2022] GET /identity/v3/role_assignments?user.id=b7bfc576d3154257a676f91304b0d7b4&scope.project.id=21c5201b37ab4a83a73a8931758d7d1a&role.id=3ab0ee91b3284a65a0dd5696a6250883 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:18.663347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee68ae6-c886-4945-a035-fa2c13069e3a None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:18.663856 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee68ae6-c886-4945-a035-fa2c13069e3a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:18.664342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee68ae6-c886-4945-a035-fa2c13069e3a None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:18.665967 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 88/181] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:18.667051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee68ae6-c886-4945-a035-fa2c13069e3a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:18.784609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079decdb-db2c-451f-a276-7fdf7dd7d44d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:18.879357 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4a067-4fbd-48f3-9ec9-dd0377d7be42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:18.880180 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a067-4fbd-48f3-9ec9-dd0377d7be42 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:18.880738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a067-4fbd-48f3-9ec9-dd0377d7be42 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:18.880738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a067-4fbd-48f3-9ec9-dd0377d7be42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:18.892038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee68ae6-c886-4945-a035-fa2c13069e3a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:18.905793 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4a067-4fbd-48f3-9ec9-dd0377d7be42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:18.914124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-cef4a067-4fbd-48f3-9ec9-dd0377d7be42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:18.914124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:18.914124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:18.914124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:18.914124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:18.914124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:18.914124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:18.914124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:18.914124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:18.914124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:18.914124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:18.914124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:18.914124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:18.914124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:18.914124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:18.914124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:18.915091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:18.915091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:18.915091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:18.915091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:18.915091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:18.915091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:18.915091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:18.915091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:18.915091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:18.915091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:18.915091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:18.915091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:18.915091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:18.915091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:18.915091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:18.915091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:18.915971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:18.915971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:18.915971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:18.915971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:18.915971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:18.915971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:18.915971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:18.915971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:18.915971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:18.915971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:18.915971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:18.915971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:18.918664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 94/182] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:52:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:18.928303 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004e8b8-fa59-4992-869f-1b4594ec0e97 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:18.928303 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004e8b8-fa59-4992-869f-1b4594ec0e97 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:18.928303 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004e8b8-fa59-4992-869f-1b4594ec0e97 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:18.928848 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:18.929260 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 95/183] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:18.936829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e02ac-3790-4572-81c8-388238dfb9df None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:18.936829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e02ac-3790-4572-81c8-388238dfb9df None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:18.936829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e02ac-3790-4572-81c8-388238dfb9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:18.956853 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e92f0-5ae2-4aef-ab0a-7ab705d7c5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:18.956853 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e92f0-5ae2-4aef-ab0a-7ab705d7c5ee None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:18.956853 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e92f0-5ae2-4aef-ab0a-7ab705d7c5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:18.956853 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e92f0-5ae2-4aef-ab0a-7ab705d7c5ee None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:18.957140 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e92f0-5ae2-4aef-ab0a-7ab705d7c5ee None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:18.957140 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e92f0-5ae2-4aef-ab0a-7ab705d7c5ee None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:18.957140 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 89/184] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:18 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:18.957140 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e92f0-5ae2-4aef-ab0a-7ab705d7c5ee None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:18.970479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-dc2e02ac-3790-4572-81c8-388238dfb9df None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:18.972695 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-dc2e02ac-3790-4572-81c8-388238dfb9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:19.053142 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:19.074288 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 96/185] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:19.082748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7672635-90e9-4e84-a9b2-e3b74b5dde4a None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:19.082748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7672635-90e9-4e84-a9b2-e3b74b5dde4a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:19.082748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7672635-90e9-4e84-a9b2-e3b74b5dde4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:19.088565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e92f0-5ae2-4aef-ab0a-7ab705d7c5ee None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:19.094604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-a7672635-90e9-4e84-a9b2-e3b74b5dde4a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:19.095841 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a7672635-90e9-4e84-a9b2-e3b74b5dde4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:19.126508 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c09c71-2992-4bb4-8bd2-b6128481fc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:19.127534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c09c71-2992-4bb4-8bd2-b6128481fc6a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:19.127827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c09c71-2992-4bb4-8bd2-b6128481fc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:19.130651 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c09c71-2992-4bb4-8bd2-b6128481fc6a None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:19.131586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c09c71-2992-4bb4-8bd2-b6128481fc6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:19.131905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c09c71-2992-4bb4-8bd2-b6128481fc6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:19.133346 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c09c71-2992-4bb4-8bd2-b6128481fc6a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:19.143119 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 90/186] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:19.144047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c09c71-2992-4bb4-8bd2-b6128481fc6a None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:19.174283 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 97/187] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:19.175993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7da64-8d3d-44ef-8cc8-4c0ba5615a3a None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:19.176284 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7da64-8d3d-44ef-8cc8-4c0ba5615a3a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:19.176562 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7da64-8d3d-44ef-8cc8-4c0ba5615a3a None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:19.177835 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 98/188] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:19.182182 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7da64-8d3d-44ef-8cc8-4c0ba5615a3a None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:19.209631 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:19.257455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c09c71-2992-4bb4-8bd2-b6128481fc6a None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:19.290915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7da64-8d3d-44ef-8cc8-4c0ba5615a3a None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:19.298696 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357922d7-4278-4ab6-98e9-92c432a7377b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:19.299374 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357922d7-4278-4ab6-98e9-92c432a7377b None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:19.299666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357922d7-4278-4ab6-98e9-92c432a7377b None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:19.299950 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357922d7-4278-4ab6-98e9-92c432a7377b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:19.300799 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357922d7-4278-4ab6-98e9-92c432a7377b None admin] RBAC: Adding query filter params (role.id=a82e24db1e3c4f868a586e122dae393e, scope.project.id=10cc8d215a4045f1b83e4120a718db23, user.id=b7bfc576d3154257a676f91304b0d7b4) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:19.301116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357922d7-4278-4ab6-98e9-92c432a7377b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:19.302622 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357922d7-4278-4ab6-98e9-92c432a7377b None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:19.308867 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 91/189] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:19 2022] GET /identity/v3/role_assignments?user.id=b7bfc576d3154257a676f91304b0d7b4&scope.project.id=10cc8d215a4045f1b83e4120a718db23&role.id=a82e24db1e3c4f868a586e122dae393e => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:19.309673 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357922d7-4278-4ab6-98e9-92c432a7377b None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:19.311660 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:19.340641 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdabc6d-ca3d-4f05-bd51-d128bf495f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:19.341499 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdabc6d-ca3d-4f05-bd51-d128bf495f47 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:19.341980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdabc6d-ca3d-4f05-bd51-d128bf495f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:19.342350 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdabc6d-ca3d-4f05-bd51-d128bf495f47 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:19.355706 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdabc6d-ca3d-4f05-bd51-d128bf495f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:19.365022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-6fdabc6d-ca3d-4f05-bd51-d128bf495f47 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:19.365022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:19.365022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:19.365022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:19.365022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:19.365022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:19.365022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:19.365022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:19.365022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:19.365022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:19.365022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:19.365022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:19.365022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:19.365022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:19.365022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:19.365022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:19.366071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:19.366071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:19.366071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:19.366071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:19.366071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:19.366071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:19.366071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:19.366071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:19.366071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:19.366071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:19.366071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:19.366071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:19.366071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:19.366071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:19.366071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:19.366071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:19.372301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:19.372301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:19.372301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:19.372301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:19.372301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:19.372301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:19.372301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:19.372301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:19.372301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:19.372301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:19.372301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:19.372301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:19.372301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 99/190] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:52:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:19.372301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934047b3-08af-497e-b2e8-d7b05f32d72a None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:19.373699 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934047b3-08af-497e-b2e8-d7b05f32d72a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:19.373699 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934047b3-08af-497e-b2e8-d7b05f32d72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:19.387816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-934047b3-08af-497e-b2e8-d7b05f32d72a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:19.389830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-934047b3-08af-497e-b2e8-d7b05f32d72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:19.448555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357922d7-4278-4ab6-98e9-92c432a7377b None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:19.475201 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:19.479208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 100/191] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:19.480686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934047b3-08af-497e-b2e8-d7b05f32d72a None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:19.493591 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcc74a-60aa-4e4f-b0f3-9b107bfaca0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:19.494494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcc74a-60aa-4e4f-b0f3-9b107bfaca0f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:19.494878 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcc74a-60aa-4e4f-b0f3-9b107bfaca0f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:19.495164 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcc74a-60aa-4e4f-b0f3-9b107bfaca0f None admin] PATH_INFO: `/v3/users/22180e0b730a4c8187f67e3fb6275b0f` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:19.511418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcc74a-60aa-4e4f-b0f3-9b107bfaca0f None admin] RBAC: Authorizing `identity:get_user(user_id=22180e0b730a4c8187f67e3fb6275b0f)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:19.512920 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcc74a-60aa-4e4f-b0f3-9b107bfaca0f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:19.518739 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 92/192] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:19 2022] GET /identity/v3/users/22180e0b730a4c8187f67e3fb6275b0f => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:19.519887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcc74a-60aa-4e4f-b0f3-9b107bfaca0f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:19.609513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934047b3-08af-497e-b2e8-d7b05f32d72a None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:19.617420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:19.637080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcc74a-60aa-4e4f-b0f3-9b107bfaca0f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:19.647156 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d5576-8b4f-4595-866f-2091f48958ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:19.647965 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d5576-8b4f-4595-866f-2091f48958ea None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:19.648244 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d5576-8b4f-4595-866f-2091f48958ea None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:19.648518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d5576-8b4f-4595-866f-2091f48958ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:19.649700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d5576-8b4f-4595-866f-2091f48958ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:19.651220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d5576-8b4f-4595-866f-2091f48958ea None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:19.654906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 101/193] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:52:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:19.656188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d5576-8b4f-4595-866f-2091f48958ea None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:19.676623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78619e8-7064-4ac5-839d-49b6c751dc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:19.677300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78619e8-7064-4ac5-839d-49b6c751dc4f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:19.677579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78619e8-7064-4ac5-839d-49b6c751dc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:19.677853 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78619e8-7064-4ac5-839d-49b6c751dc4f None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:19.678634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78619e8-7064-4ac5-839d-49b6c751dc4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:19.678943 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78619e8-7064-4ac5-839d-49b6c751dc4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:19.680359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78619e8-7064-4ac5-839d-49b6c751dc4f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:19.689465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 93/194] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:52:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:19.690379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78619e8-7064-4ac5-839d-49b6c751dc4f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:19.738200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:19.782106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d5576-8b4f-4595-866f-2091f48958ea None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:19.800945 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78619e8-7064-4ac5-839d-49b6c751dc4f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:19.820317 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e0bea-26f5-4401-8350-8f392cf438bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:19.821126 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e0bea-26f5-4401-8350-8f392cf438bb None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:19.821411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e0bea-26f5-4401-8350-8f392cf438bb None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:19.821685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e0bea-26f5-4401-8350-8f392cf438bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:19.833269 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e0bea-26f5-4401-8350-8f392cf438bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:19.835915 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494fc28-80f8-422c-a4ca-b02e9dc9e599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:19.836576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494fc28-80f8-422c-a4ca-b02e9dc9e599 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:19.836858 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494fc28-80f8-422c-a4ca-b02e9dc9e599 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:19.837133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494fc28-80f8-422c-a4ca-b02e9dc9e599 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:19.841061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-522e0bea-26f5-4401-8350-8f392cf438bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:19.841061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:19.841061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:19.841061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:19.841061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:19.841061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:19.841061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:19.841061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:19.841061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:19.841061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:19.841061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:19.841061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:19.841061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:19.841061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:19.841061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:19.841061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:19.842112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:19.842112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:19.842112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:19.842112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:19.842112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:19.842112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:19.842112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:19.842112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:19.842112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:19.842112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:19.842112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:19.842112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:19.842112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:19.842112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:19.842112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:19.842112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:19.842991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:19.842991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:19.842991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:19.842991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:19.842991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:19.842991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:19.842991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:19.842991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:19.842991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:19.842991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:19.842991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:19.842991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:19.843630 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 102/195] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:19.844042 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e0bea-26f5-4401-8350-8f392cf438bb None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:19.878702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:19.911527 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494fc28-80f8-422c-a4ca-b02e9dc9e599 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:19.913020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494fc28-80f8-422c-a4ca-b02e9dc9e599 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:19.949410 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e0bea-26f5-4401-8350-8f392cf438bb None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:19.957577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:19.986784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc416d-50a1-415e-aa38-c11bc99efbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:19.987573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc416d-50a1-415e-aa38-c11bc99efbd3 None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:19.987858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc416d-50a1-415e-aa38-c11bc99efbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:19.988137 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc416d-50a1-415e-aa38-c11bc99efbd3 None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:19.988858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc416d-50a1-415e-aa38-c11bc99efbd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:19.990466 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc416d-50a1-415e-aa38-c11bc99efbd3 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:20.024037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 94/196] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:20.024897 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494fc28-80f8-422c-a4ca-b02e9dc9e599 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:20.094369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 103/197] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:52:19 2022] POST /identity/v3/users => generated 297 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:52:20.095570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc416d-50a1-415e-aa38-c11bc99efbd3 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:20.144478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494fc28-80f8-422c-a4ca-b02e9dc9e599 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:20.189671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07600882-a0d1-4d9d-8d98-683d3c06e971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:20.191076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07600882-a0d1-4d9d-8d98-683d3c06e971 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:20.191356 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07600882-a0d1-4d9d-8d98-683d3c06e971 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:20.191632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07600882-a0d1-4d9d-8d98-683d3c06e971 None admin] PATH_INFO: `/v3/users/22180e0b730a4c8187f67e3fb6275b0f` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:20.192949 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07600882-a0d1-4d9d-8d98-683d3c06e971 None admin] RBAC: Authorizing `identity:get_user(user_id=22180e0b730a4c8187f67e3fb6275b0f)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:20.194765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07600882-a0d1-4d9d-8d98-683d3c06e971 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:20.196520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 95/198] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:20 2022] GET /identity/v3/users/22180e0b730a4c8187f67e3fb6275b0f => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:20.197381 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07600882-a0d1-4d9d-8d98-683d3c06e971 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:20.214305 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:20.214566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:20.232722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc416d-50a1-415e-aa38-c11bc99efbd3 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:20.276788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015c230-1be0-48d4-9e68-98188c040160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:20.276788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015c230-1be0-48d4-9e68-98188c040160 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:20.276788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015c230-1be0-48d4-9e68-98188c040160 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:20.276788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015c230-1be0-48d4-9e68-98188c040160 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:20.277238 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015c230-1be0-48d4-9e68-98188c040160 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:20.277308 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015c230-1be0-48d4-9e68-98188c040160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:20.278716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015c230-1be0-48d4-9e68-98188c040160 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:20.288519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 104/199] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:20.289164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015c230-1be0-48d4-9e68-98188c040160 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:20.321866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07600882-a0d1-4d9d-8d98-683d3c06e971 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:20.376256 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78a87b-01ce-40d2-8b61-2a21289d954b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:20.377137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78a87b-01ce-40d2-8b61-2a21289d954b None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:20.377423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78a87b-01ce-40d2-8b61-2a21289d954b None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:20.377696 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78a87b-01ce-40d2-8b61-2a21289d954b None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:20.379063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78a87b-01ce-40d2-8b61-2a21289d954b None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:20.380547 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78a87b-01ce-40d2-8b61-2a21289d954b None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:20.382294 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 96/200] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:20 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:20.384133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78a87b-01ce-40d2-8b61-2a21289d954b None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:20.416673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015c230-1be0-48d4-9e68-98188c040160 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:20.464917 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1106f-121e-4994-9d91-54a1f2a1b736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:20.465584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1106f-121e-4994-9d91-54a1f2a1b736 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:20.465872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1106f-121e-4994-9d91-54a1f2a1b736 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:20.466156 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1106f-121e-4994-9d91-54a1f2a1b736 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:20.495609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:20.497628 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:20.526585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78a87b-01ce-40d2-8b61-2a21289d954b None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:20.561399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1106f-121e-4994-9d91-54a1f2a1b736 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:20.562941 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1106f-121e-4994-9d91-54a1f2a1b736 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:20.578431 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782461db-f9a7-4b98-8b88-15152c2e3fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:20.580047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782461db-f9a7-4b98-8b88-15152c2e3fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:20.581070 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782461db-f9a7-4b98-8b88-15152c2e3fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:20.581708 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782461db-f9a7-4b98-8b88-15152c2e3fe0 None admin] PATH_INFO: `/v3/users/124e6c742e5f441d862efe694b44d923` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:20.583710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782461db-f9a7-4b98-8b88-15152c2e3fe0 None admin] RBAC: Authorizing `identity:get_user(user_id=124e6c742e5f441d862efe694b44d923)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:20.585990 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782461db-f9a7-4b98-8b88-15152c2e3fe0 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:20.588993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 97/201] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:20 2022] GET /identity/v3/users/124e6c742e5f441d862efe694b44d923 => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:20.590091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782461db-f9a7-4b98-8b88-15152c2e3fe0 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:20.692428 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 105/202] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:20.692953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1106f-121e-4994-9d91-54a1f2a1b736 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:20.744251 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782461db-f9a7-4b98-8b88-15152c2e3fe0 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:20.756023 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:20.799124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f0b20-6d0c-4e45-96c6-9238719ac96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:20.800699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f0b20-6d0c-4e45-96c6-9238719ac96e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:20.801164 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f0b20-6d0c-4e45-96c6-9238719ac96e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:20.802068 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f0b20-6d0c-4e45-96c6-9238719ac96e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:20.803375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f0b20-6d0c-4e45-96c6-9238719ac96e None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=cc9a9fe967674703a9776dca128bc387, user.id=22180e0b730a4c8187f67e3fb6275b0f) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:20.804736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f0b20-6d0c-4e45-96c6-9238719ac96e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:20.807323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f0b20-6d0c-4e45-96c6-9238719ac96e None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:20.812623 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1106f-121e-4994-9d91-54a1f2a1b736 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:20.816360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 98/203] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:20 2022] GET /identity/v3/role_assignments?user.id=22180e0b730a4c8187f67e3fb6275b0f&scope.project.id=cc9a9fe967674703a9776dca128bc387&role.id=5891920310b54787bd1152d736d2a3f1 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:20.817222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f0b20-6d0c-4e45-96c6-9238719ac96e None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:20.832064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:20.858738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a710347-cdb8-4159-a1fd-af2a5d5b0fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:20.859411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a710347-cdb8-4159-a1fd-af2a5d5b0fac None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:20.859958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a710347-cdb8-4159-a1fd-af2a5d5b0fac None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:20.860421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a710347-cdb8-4159-a1fd-af2a5d5b0fac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:20.968378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f0b20-6d0c-4e45-96c6-9238719ac96e None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:21.014495 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b08d9-ad7c-44a4-ba64-fef3fe67d563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:21.015424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b08d9-ad7c-44a4-ba64-fef3fe67d563 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:21.015707 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b08d9-ad7c-44a4-ba64-fef3fe67d563 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:21.016047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b08d9-ad7c-44a4-ba64-fef3fe67d563 None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:21.017425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b08d9-ad7c-44a4-ba64-fef3fe67d563 None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:21.019049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b08d9-ad7c-44a4-ba64-fef3fe67d563 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:21.021428 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 99/204] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:20 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:21.031289 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b08d9-ad7c-44a4-ba64-fef3fe67d563 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:21.040628 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a710347-cdb8-4159-a1fd-af2a5d5b0fac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:21.042560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a710347-cdb8-4159-a1fd-af2a5d5b0fac None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:21.128523 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:21.162599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b08d9-ad7c-44a4-ba64-fef3fe67d563 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:21.190612 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:21.210373 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecdbb9-7d23-4eb1-8336-e78a47088929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:21.211869 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecdbb9-7d23-4eb1-8336-e78a47088929 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:21.212530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecdbb9-7d23-4eb1-8336-e78a47088929 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:21.213033 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecdbb9-7d23-4eb1-8336-e78a47088929 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:21.214237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ecdbb9-7d23-4eb1-8336-e78a47088929 None admin] RBAC: Adding query filter params (role.id=a82e24db1e3c4f868a586e122dae393e, scope.project.id=cc9a9fe967674703a9776dca128bc387, user.id=22180e0b730a4c8187f67e3fb6275b0f) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:21.214988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ecdbb9-7d23-4eb1-8336-e78a47088929 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:21.217725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ecdbb9-7d23-4eb1-8336-e78a47088929 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:21.225235 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 100/205] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:21 2022] GET /identity/v3/role_assignments?user.id=22180e0b730a4c8187f67e3fb6275b0f&scope.project.id=cc9a9fe967674703a9776dca128bc387&role.id=a82e24db1e3c4f868a586e122dae393e => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:21.273889 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 106/206] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:21.280438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecdbb9-7d23-4eb1-8336-e78a47088929 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:21.439881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecdbb9-7d23-4eb1-8336-e78a47088929 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:21.488689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3c3df-5376-498f-881b-6942d47ce801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:21.489581 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3c3df-5376-498f-881b-6942d47ce801 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:21.489870 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3c3df-5376-498f-881b-6942d47ce801 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:21.490149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3c3df-5376-498f-881b-6942d47ce801 None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:21.491420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3c3df-5376-498f-881b-6942d47ce801 None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:21.492898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3c3df-5376-498f-881b-6942d47ce801 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:21.494667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 101/207] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:21 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:21.501639 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a710347-cdb8-4159-a1fd-af2a5d5b0fac None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:21.624962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a710347-cdb8-4159-a1fd-af2a5d5b0fac None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:21.660986 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba40a0a-7a45-4336-8434-0ce983123542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:21.661709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba40a0a-7a45-4336-8434-0ce983123542 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:21.662008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba40a0a-7a45-4336-8434-0ce983123542 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:21.662295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba40a0a-7a45-4336-8434-0ce983123542 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:21.663204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba40a0a-7a45-4336-8434-0ce983123542 None admin] RBAC: Adding query filter params (role.id=192149ed0eab4f039954e4617e7ee779, scope.project.id=10cc8d215a4045f1b83e4120a718db23, user.id=124e6c742e5f441d862efe694b44d923) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:21.663523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba40a0a-7a45-4336-8434-0ce983123542 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:21.665013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba40a0a-7a45-4336-8434-0ce983123542 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:21.672815 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 107/208] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:21 2022] GET /identity/v3/role_assignments?user.id=124e6c742e5f441d862efe694b44d923&scope.project.id=10cc8d215a4045f1b83e4120a718db23&role.id=192149ed0eab4f039954e4617e7ee779 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:22.675425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e97859-a471-45c7-b582-c26ebb9f55be None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:22.675984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e97859-a471-45c7-b582-c26ebb9f55be None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:22.675984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e97859-a471-45c7-b582-c26ebb9f55be None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:22.677684 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 102/209] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:22.683560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce38fb2-3a7a-4885-8621-a1cb436c79f4 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:22.684511 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce38fb2-3a7a-4885-8621-a1cb436c79f4 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:22.685141 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce38fb2-3a7a-4885-8621-a1cb436c79f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:22.704659 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-dce38fb2-3a7a-4885-8621-a1cb436c79f4 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:22.706274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-dce38fb2-3a7a-4885-8621-a1cb436c79f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:22.719403 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:22.738076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895611f4-47d2-40ca-816d-ce0777c663f3 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:22.743548 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895611f4-47d2-40ca-816d-ce0777c663f3 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:22.743548 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895611f4-47d2-40ca-816d-ce0777c663f3 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:22.743548 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 103/210] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:22.745689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08e6f0-76b5-4265-981f-420981fb49e2 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:22.745988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08e6f0-76b5-4265-981f-420981fb49e2 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:22.746271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08e6f0-76b5-4265-981f-420981fb49e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:22.769883 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-ac08e6f0-76b5-4265-981f-420981fb49e2 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:22.771827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ac08e6f0-76b5-4265-981f-420981fb49e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:22.774644 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:22.797335 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 108/211] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:22.821616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce38fb2-3a7a-4885-8621-a1cb436c79f4 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:22.862273 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 104/212] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:22.892935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08e6f0-76b5-4265-981f-420981fb49e2 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:22.949143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce38fb2-3a7a-4885-8621-a1cb436c79f4 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:22.996529 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffaa84-8acd-48a4-9c0b-03450434a2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:22.997525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffaa84-8acd-48a4-9c0b-03450434a2df None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:23.000400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffaa84-8acd-48a4-9c0b-03450434a2df None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:23.001032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffaa84-8acd-48a4-9c0b-03450434a2df None admin] PATH_INFO: `/v3/roles/member` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:23.016103 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08e6f0-76b5-4265-981f-420981fb49e2 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:23.017327 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ffaa84-8acd-48a4-9c0b-03450434a2df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:23.026966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-05ffaa84-8acd-48a4-9c0b-03450434a2df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:23.026966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:23.026966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:23.026966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:23.026966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:23.026966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:23.026966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:23.026966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:23.026966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:23.026966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:23.026966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:23.026966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:23.026966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:23.026966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:23.026966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:23.026966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:23.027954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:23.027954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:23.027954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:23.027954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:23.027954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:23.027954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:23.027954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:23.027954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:23.027954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:23.027954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:23.027954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:23.027954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:23.027954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:23.027954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:23.027954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:23.027954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:23.028780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:23.028780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:23.028780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:23.028780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:23.028780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:23.028780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:23.028780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:23.028780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:23.028780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:23.028780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:23.028780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:23.028780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:23.028780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 109/213] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:52:23.035153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffaa84-8acd-48a4-9c0b-03450434a2df None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:23.048908 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:23.068198 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a329c-7006-4bf6-9575-89657576fed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:23.069512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a329c-7006-4bf6-9575-89657576fed0 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:23.070020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a329c-7006-4bf6-9575-89657576fed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:23.070515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a329c-7006-4bf6-9575-89657576fed0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:23.072031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a329c-7006-4bf6-9575-89657576fed0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:23.073766 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a329c-7006-4bf6-9575-89657576fed0 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:23.075887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 105/214] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:52:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:23.081725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a329c-7006-4bf6-9575-89657576fed0 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:23.092973 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:23.165496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffaa84-8acd-48a4-9c0b-03450434a2df None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:23.214743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391713f5-91d7-4a70-9f76-8e49e74141d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:23.215337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391713f5-91d7-4a70-9f76-8e49e74141d1 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:23.215628 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391713f5-91d7-4a70-9f76-8e49e74141d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:23.215974 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391713f5-91d7-4a70-9f76-8e49e74141d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:23.217296 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391713f5-91d7-4a70-9f76-8e49e74141d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:23.217609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391713f5-91d7-4a70-9f76-8e49e74141d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:23.219045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391713f5-91d7-4a70-9f76-8e49e74141d1 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:23.222227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a329c-7006-4bf6-9575-89657576fed0 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:23.228609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 110/215] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:23.230298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04e7ee-479f-496b-b454-37439cac3daf None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:23.230801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04e7ee-479f-496b-b454-37439cac3daf None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:23.231080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04e7ee-479f-496b-b454-37439cac3daf None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:23.233275 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 111/216] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:23.238684 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04e7ee-479f-496b-b454-37439cac3daf None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:23.267735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92138677-3560-4620-b335-958f3ae24188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:23.268817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92138677-3560-4620-b335-958f3ae24188 None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:23.269354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92138677-3560-4620-b335-958f3ae24188 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:23.269905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92138677-3560-4620-b335-958f3ae24188 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:23.270943 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92138677-3560-4620-b335-958f3ae24188 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:23.272610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92138677-3560-4620-b335-958f3ae24188 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:23.315106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:23.349560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04e7ee-479f-496b-b454-37439cac3daf None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:23.351901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 106/217] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:52:23 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:52:23.353197 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecd901-8f95-4261-b444-82ac3aa11f03 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:23.353497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecd901-8f95-4261-b444-82ac3aa11f03 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:23.353781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecd901-8f95-4261-b444-82ac3aa11f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:23.380051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-72ecd901-8f95-4261-b444-82ac3aa11f03 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:23.382137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-72ecd901-8f95-4261-b444-82ac3aa11f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:23.409939 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244eaf2-d6f3-4f25-ae49-9989c3b8b0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:23.410972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244eaf2-d6f3-4f25-ae49-9989c3b8b0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:23.411191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244eaf2-d6f3-4f25-ae49-9989c3b8b0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:23.411481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244eaf2-d6f3-4f25-ae49-9989c3b8b0c4 None admin] PATH_INFO: `/v3/users/e0fa7d63fd914f3082862d1181b78157` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:23.435007 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244eaf2-d6f3-4f25-ae49-9989c3b8b0c4 None admin] RBAC: Authorizing `identity:get_user(user_id=e0fa7d63fd914f3082862d1181b78157)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:23.437227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244eaf2-d6f3-4f25-ae49-9989c3b8b0c4 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:23.439567 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 112/218] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:23 2022] GET /identity/v3/users/e0fa7d63fd914f3082862d1181b78157 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:23.447664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244eaf2-d6f3-4f25-ae49-9989c3b8b0c4 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:23.486513 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:23.505543 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 107/219] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:23.530418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecd901-8f95-4261-b444-82ac3aa11f03 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:23.592175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244eaf2-d6f3-4f25-ae49-9989c3b8b0c4 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:23.620997 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:23.634527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ec149-0bb0-459d-ab86-7290adbc2d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:23.635591 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ec149-0bb0-459d-ab86-7290adbc2d67 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:23.636062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ec149-0bb0-459d-ab86-7290adbc2d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:23.636526 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ec149-0bb0-459d-ab86-7290adbc2d67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:23.679983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecd901-8f95-4261-b444-82ac3aa11f03 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:23.732888 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf88ea6-cfd8-4762-a7d2-625190a6825d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:23.733729 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf88ea6-cfd8-4762-a7d2-625190a6825d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:23.734016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf88ea6-cfd8-4762-a7d2-625190a6825d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:23.734297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf88ea6-cfd8-4762-a7d2-625190a6825d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:23.740281 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ec149-0bb0-459d-ab86-7290adbc2d67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:23.741715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ec149-0bb0-459d-ab86-7290adbc2d67 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:23.751655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf88ea6-cfd8-4762-a7d2-625190a6825d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:23.761052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-faf88ea6-cfd8-4762-a7d2-625190a6825d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:23.761052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:23.761052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:23.761052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:23.761052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:23.761052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:23.761052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:23.761052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:23.761052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:23.761052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:23.761052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:23.761052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:23.761052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:23.761052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:23.761052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:23.761052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:23.762028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:23.762028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:23.762028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:23.762028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:23.762028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:23.762028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:23.762028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:23.762028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:23.762028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:23.762028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:23.762028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:23.762028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:23.762028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:23.762028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:23.762028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:23.762028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:23.762982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:23.762982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:23.762982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:23.762982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:23.762982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:23.762982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:23.762982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:23.762982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:23.762982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:23.762982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:23.762982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:23.762982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:23.762982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 108/220] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:52:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:52:23.764941 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a4792-44c1-4302-9f6b-7de8af7a64dc None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:23.764941 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a4792-44c1-4302-9f6b-7de8af7a64dc None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:23.764941 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a4792-44c1-4302-9f6b-7de8af7a64dc None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:23.766566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 109/221] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:23.769387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a4792-44c1-4302-9f6b-7de8af7a64dc None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:23.844048 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:23.868563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 113/222] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:23.870667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f5a04-8370-4ce4-a420-b24fe8306ec3 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:23.870667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f5a04-8370-4ce4-a420-b24fe8306ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:23.870667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f5a04-8370-4ce4-a420-b24fe8306ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:23.894203 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-6e2f5a04-8370-4ce4-a420-b24fe8306ec3 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:23.895832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6e2f5a04-8370-4ce4-a420-b24fe8306ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:23.907534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a4792-44c1-4302-9f6b-7de8af7a64dc None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:23.938934 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:23.945570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6f6eb-abc1-47df-ad9e-a98dab0e82c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:23.946260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6f6eb-abc1-47df-ad9e-a98dab0e82c8 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:23.946564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6f6eb-abc1-47df-ad9e-a98dab0e82c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:23.946873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6f6eb-abc1-47df-ad9e-a98dab0e82c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:23.948044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6f6eb-abc1-47df-ad9e-a98dab0e82c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:23.948361 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6f6eb-abc1-47df-ad9e-a98dab0e82c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:23.951176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6f6eb-abc1-47df-ad9e-a98dab0e82c8 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:23.962754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 110/223] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:23.968248 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6f6eb-abc1-47df-ad9e-a98dab0e82c8 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:23.983167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 114/224] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:23.994779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f5a04-8370-4ce4-a420-b24fe8306ec3 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:24.110723 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6f6eb-abc1-47df-ad9e-a98dab0e82c8 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:24.127916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:24.128345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f5a04-8370-4ce4-a420-b24fe8306ec3 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:24.150566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36743c-363c-4222-854c-a9ceeb456c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:24.151844 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36743c-363c-4222-854c-a9ceeb456c03 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:24.152342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36743c-363c-4222-854c-a9ceeb456c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:24.152854 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36743c-363c-4222-854c-a9ceeb456c03 None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:24.154570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36743c-363c-4222-854c-a9ceeb456c03 None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:24.156787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36743c-363c-4222-854c-a9ceeb456c03 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:24.159565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 111/225] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:23 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:24.160788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36743c-363c-4222-854c-a9ceeb456c03 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:24.175728 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98c5dd-50a1-4182-a315-031a04583fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:24.176667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98c5dd-50a1-4182-a315-031a04583fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:24.176980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98c5dd-50a1-4182-a315-031a04583fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:24.177268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98c5dd-50a1-4182-a315-031a04583fd6 None admin] PATH_INFO: `/v3/users/22180e0b730a4c8187f67e3fb6275b0f` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:24.178811 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98c5dd-50a1-4182-a315-031a04583fd6 None admin] RBAC: Authorizing `identity:get_user(user_id=22180e0b730a4c8187f67e3fb6275b0f)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:24.181134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98c5dd-50a1-4182-a315-031a04583fd6 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:24.183662 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 115/226] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:23 2022] GET /identity/v3/users/22180e0b730a4c8187f67e3fb6275b0f => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:24.189870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98c5dd-50a1-4182-a315-031a04583fd6 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:24.225940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:24.282329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36743c-363c-4222-854c-a9ceeb456c03 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:24.322735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc62b1-2ac5-40cb-99bd-f16a6a37ae92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:24.324147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc62b1-2ac5-40cb-99bd-f16a6a37ae92 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:24.324700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc62b1-2ac5-40cb-99bd-f16a6a37ae92 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:24.325288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc62b1-2ac5-40cb-99bd-f16a6a37ae92 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:24.329770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98c5dd-50a1-4182-a315-031a04583fd6 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:24.338157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc62b1-2ac5-40cb-99bd-f16a6a37ae92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:24.352463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-e5fc62b1-2ac5-40cb-99bd-f16a6a37ae92 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:24.352463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:24.352463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:24.352463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:24.352463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:24.352463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:24.352463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:24.352463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:24.352463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:24.352463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:24.352463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:24.352463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:24.352463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:24.352463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:24.352463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:24.352463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:24.353445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:24.353445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:24.353445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:24.353445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:24.353445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:24.353445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:24.353445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:24.353445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:24.353445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:24.353445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:24.353445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:24.353445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:24.353445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:24.353445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:24.353445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:24.353445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:24.354241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:24.354241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:24.354241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:24.354241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:24.354241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:24.354241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:24.354241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:24.354241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:24.354241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:24.354241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:24.354241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:24.354241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:24.356740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 112/227] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:52:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:24.359004 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc62b1-2ac5-40cb-99bd-f16a6a37ae92 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:24.373816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf8004-b619-43ee-8c86-d198556e5b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:24.375037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf8004-b619-43ee-8c86-d198556e5b27 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:24.375536 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf8004-b619-43ee-8c86-d198556e5b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:24.376020 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf8004-b619-43ee-8c86-d198556e5b27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:24.377125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf8004-b619-43ee-8c86-d198556e5b27 None admin] RBAC: Adding query filter params (role.id=3ab0ee91b3284a65a0dd5696a6250883, scope.project.id=cc9a9fe967674703a9776dca128bc387, user.id=e0fa7d63fd914f3082862d1181b78157) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:24.377649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf8004-b619-43ee-8c86-d198556e5b27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:24.379378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf8004-b619-43ee-8c86-d198556e5b27 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:24.386364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 116/228] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:24 2022] GET /identity/v3/role_assignments?user.id=e0fa7d63fd914f3082862d1181b78157&scope.project.id=cc9a9fe967674703a9776dca128bc387&role.id=3ab0ee91b3284a65a0dd5696a6250883 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:24.388016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf8004-b619-43ee-8c86-d198556e5b27 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:24.426721 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:24.490691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc62b1-2ac5-40cb-99bd-f16a6a37ae92 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:24.542064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1269a9e-7ad7-4ac2-860b-fecdde4f6608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:24.545194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1269a9e-7ad7-4ac2-860b-fecdde4f6608 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:24.545651 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1269a9e-7ad7-4ac2-860b-fecdde4f6608 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:24.546044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1269a9e-7ad7-4ac2-860b-fecdde4f6608 None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:24.553774 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1269a9e-7ad7-4ac2-860b-fecdde4f6608 None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:24.555578 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1269a9e-7ad7-4ac2-860b-fecdde4f6608 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:24.557719 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 113/229] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:24 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:24.565603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1269a9e-7ad7-4ac2-860b-fecdde4f6608 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:24.601154 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:24.618619 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf8004-b619-43ee-8c86-d198556e5b27 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:24.693605 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1269a9e-7ad7-4ac2-860b-fecdde4f6608 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:24.706968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:24.715284 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc520e3-6365-460f-9852-2e4aed237e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:24.715991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc520e3-6365-460f-9852-2e4aed237e3f None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:24.716307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc520e3-6365-460f-9852-2e4aed237e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:24.716655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc520e3-6365-460f-9852-2e4aed237e3f None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:24.717423 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc520e3-6365-460f-9852-2e4aed237e3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:24.717988 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc520e3-6365-460f-9852-2e4aed237e3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:24.719260 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc520e3-6365-460f-9852-2e4aed237e3f None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:24.734496 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2a0df-fbc2-4bf0-b9a2-0e95342dfc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:24.735810 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2a0df-fbc2-4bf0-b9a2-0e95342dfc8f None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:24.736099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2a0df-fbc2-4bf0-b9a2-0e95342dfc8f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:24.736512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2a0df-fbc2-4bf0-b9a2-0e95342dfc8f None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387/users/22180e0b730a4c8187f67e3fb6275b0f/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:24.739069 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2a0df-fbc2-4bf0-b9a2-0e95342dfc8f None admin] RBAC: Authorizing `identity:create_grant(project_id=cc9a9fe967674703a9776dca128bc387, user_id=22180e0b730a4c8187f67e3fb6275b0f, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:24.740683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2a0df-fbc2-4bf0-b9a2-0e95342dfc8f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:24.745007 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 117/230] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:52:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:24.754970 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc520e3-6365-460f-9852-2e4aed237e3f None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:24.819074 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 114/231] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:52:24 2022] PUT /identity/v3/projects/cc9a9fe967674703a9776dca128bc387/users/22180e0b730a4c8187f67e3fb6275b0f/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:52:24.914516 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc520e3-6365-460f-9852-2e4aed237e3f None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:24.959920 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45083962-c0a2-4d79-a44b-d40a8288d29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:24.961159 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45083962-c0a2-4d79-a44b-d40a8288d29d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:24.961459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45083962-c0a2-4d79-a44b-d40a8288d29d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:24.961743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45083962-c0a2-4d79-a44b-d40a8288d29d None admin] PATH_INFO: `/v3/users/22180e0b730a4c8187f67e3fb6275b0f` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:24.963470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45083962-c0a2-4d79-a44b-d40a8288d29d None admin] RBAC: Authorizing `identity:get_user(user_id=22180e0b730a4c8187f67e3fb6275b0f)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:24.964987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45083962-c0a2-4d79-a44b-d40a8288d29d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:24.968146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 118/232] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:24 2022] GET /identity/v3/users/22180e0b730a4c8187f67e3fb6275b0f => generated 283 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:24.975773 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2a0df-fbc2-4bf0-b9a2-0e95342dfc8f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:25.155984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2a0df-fbc2-4bf0-b9a2-0e95342dfc8f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:25.206133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d0d54-495c-4fdd-abd1-e9b1a4e12e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:25.207530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d0d54-495c-4fdd-abd1-e9b1a4e12e37 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:25.208082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d0d54-495c-4fdd-abd1-e9b1a4e12e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:25.208570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d0d54-495c-4fdd-abd1-e9b1a4e12e37 None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:25.210192 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d0d54-495c-4fdd-abd1-e9b1a4e12e37 None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:25.212093 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d0d54-495c-4fdd-abd1-e9b1a4e12e37 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:25.214769 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 115/233] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:24 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:25.221480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45083962-c0a2-4d79-a44b-d40a8288d29d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:25.251706 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:25.424900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45083962-c0a2-4d79-a44b-d40a8288d29d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:25.482527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023bedce-e5a2-4edd-9ea4-0ba82d80d9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:25.483485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bedce-e5a2-4edd-9ea4-0ba82d80d9a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:25.483817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bedce-e5a2-4edd-9ea4-0ba82d80d9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:25.484147 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bedce-e5a2-4edd-9ea4-0ba82d80d9a3 None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387/users/22180e0b730a4c8187f67e3fb6275b0f/roles/a82e24db1e3c4f868a586e122dae393e` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:25.489223 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023bedce-e5a2-4edd-9ea4-0ba82d80d9a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc9a9fe967674703a9776dca128bc387, user_id=22180e0b730a4c8187f67e3fb6275b0f, role_id=a82e24db1e3c4f868a586e122dae393e)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:25.490928 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023bedce-e5a2-4edd-9ea4-0ba82d80d9a3 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:25.597836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 119/234] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:52:25 2022] PUT /identity/v3/projects/cc9a9fe967674703a9776dca128bc387/users/22180e0b730a4c8187f67e3fb6275b0f/roles/a82e24db1e3c4f868a586e122dae393e => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:52:25.934477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b48375-e0f1-4652-aecd-1c4666183953 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:25.935561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b48375-e0f1-4652-aecd-1c4666183953 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:25.935820 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b48375-e0f1-4652-aecd-1c4666183953 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:25.937383 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 116/235] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:25.944193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7912063-4e3b-4765-8a65-88879b13cb0e None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:25.944672 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7912063-4e3b-4765-8a65-88879b13cb0e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:25.944746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7912063-4e3b-4765-8a65-88879b13cb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:25.964850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-c7912063-4e3b-4765-8a65-88879b13cb0e None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:25.966123 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c7912063-4e3b-4765-8a65-88879b13cb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:25.970374 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f49a5-3ef4-4aca-8a89-c9113f7f85e1 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:25.970691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f49a5-3ef4-4aca-8a89-c9113f7f85e1 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:25.970975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f49a5-3ef4-4aca-8a89-c9113f7f85e1 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:25.972635 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 117/236] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:52:25.977791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd21a7-791f-4932-9114-1694567ba760 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:25.978077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd21a7-791f-4932-9114-1694567ba760 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:25.978357 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd21a7-791f-4932-9114-1694567ba760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:26.010207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-d9fd21a7-791f-4932-9114-1694567ba760 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:26.011922 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d9fd21a7-791f-4932-9114-1694567ba760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:26.066078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:26.109407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 120/237] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:26.134989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7912063-4e3b-4765-8a65-88879b13cb0e None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:26.141769 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 118/238] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:52:26.158942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:26.192577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd21a7-791f-4932-9114-1694567ba760 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:26.305787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7912063-4e3b-4765-8a65-88879b13cb0e None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:26.345769 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365c55e-21b7-469a-b565-f6cee8394766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:26.346608 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365c55e-21b7-469a-b565-f6cee8394766 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:26.347165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365c55e-21b7-469a-b565-f6cee8394766 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:26.347511 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365c55e-21b7-469a-b565-f6cee8394766 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:26.360254 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e365c55e-21b7-469a-b565-f6cee8394766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:26.376733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-e365c55e-21b7-469a-b565-f6cee8394766 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:26.376733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:26.376733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:26.376733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:26.376733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:26.376733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:26.376733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:26.376733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:26.376733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:26.376733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:26.376733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:26.376733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:26.376733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:26.376733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:26.376733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:26.376733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:26.377717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:26.377717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:26.377717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:26.377717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:26.377717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:26.377717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:26.377717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:26.377717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:26.377717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:26.377717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:26.377717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:26.377717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:26.377717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:26.377717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:26.377717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:26.377717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:26.378564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:26.378564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:26.378564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:26.378564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:26.378564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:26.378564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:26.378564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:26.378564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:26.378564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:26.378564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:26.378564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:26.378564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:26.378564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 121/239] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:52:26.385348 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365c55e-21b7-469a-b565-f6cee8394766 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:26.454023 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd21a7-791f-4932-9114-1694567ba760 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:26.482300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:26.530588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac708c31-bfa8-4062-9e32-59d8da41a9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:26.531403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac708c31-bfa8-4062-9e32-59d8da41a9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:26.531690 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac708c31-bfa8-4062-9e32-59d8da41a9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:26.531963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac708c31-bfa8-4062-9e32-59d8da41a9e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:26.534505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac708c31-bfa8-4062-9e32-59d8da41a9e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:26.536016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac708c31-bfa8-4062-9e32-59d8da41a9e9 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:26.537686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 119/240] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:52:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:26.543469 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac708c31-bfa8-4062-9e32-59d8da41a9e9 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:26.553844 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365c55e-21b7-469a-b565-f6cee8394766 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:26.630875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:26.646378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6978d3a-60f9-4729-9e79-04f39c81feba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:26.647090 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6978d3a-60f9-4729-9e79-04f39c81feba None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:26.647373 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6978d3a-60f9-4729-9e79-04f39c81feba None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:26.647823 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6978d3a-60f9-4729-9e79-04f39c81feba None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:26.648587 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6978d3a-60f9-4729-9e79-04f39c81feba None admin] RBAC: Adding query filter params (name=reader) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:26.648894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6978d3a-60f9-4729-9e79-04f39c81feba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:26.650311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6978d3a-60f9-4729-9e79-04f39c81feba None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:26.660821 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 122/241] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:26.666324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6978d3a-60f9-4729-9e79-04f39c81feba None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:26.789085 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac708c31-bfa8-4062-9e32-59d8da41a9e9 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:26.828847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6978d3a-60f9-4729-9e79-04f39c81feba None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:26.866651 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6196ca8-5a39-478f-9e08-c816411c51e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:26.867748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6196ca8-5a39-478f-9e08-c816411c51e5 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:26.868042 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6196ca8-5a39-478f-9e08-c816411c51e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:26.868329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6196ca8-5a39-478f-9e08-c816411c51e5 None admin] PATH_INFO: `/v3/users/464b5888738c43d1883e15171a05710c` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:26.882994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:26.887271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6196ca8-5a39-478f-9e08-c816411c51e5 None admin] RBAC: Authorizing `identity:get_user(user_id=464b5888738c43d1883e15171a05710c)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:26.890715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6196ca8-5a39-478f-9e08-c816411c51e5 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:26.892901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 123/242] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:26 2022] GET /identity/v3/users/464b5888738c43d1883e15171a05710c => generated 297 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:26.898560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6196ca8-5a39-478f-9e08-c816411c51e5 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:26.910326 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268da7b-4179-4c15-b774-94559ca9a884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:26.911054 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268da7b-4179-4c15-b774-94559ca9a884 None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:26.911368 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268da7b-4179-4c15-b774-94559ca9a884 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:26.911648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268da7b-4179-4c15-b774-94559ca9a884 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:26.912389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268da7b-4179-4c15-b774-94559ca9a884 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:26.915676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268da7b-4179-4c15-b774-94559ca9a884 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:27.026689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6196ca8-5a39-478f-9e08-c816411c51e5 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:27.028792 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 120/243] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:52:26 2022] POST /identity/v3/users => generated 293 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:52:27.034104 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bd3c8-5a08-4e8c-abab-94cfdf674127 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:27.034438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bd3c8-5a08-4e8c-abab-94cfdf674127 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:27.034721 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bd3c8-5a08-4e8c-abab-94cfdf674127 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:27.036432 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 121/244] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:27.045499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736d0df-bf0d-4d97-bc45-750d84fea77c None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:27.045958 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736d0df-bf0d-4d97-bc45-750d84fea77c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:27.046326 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736d0df-bf0d-4d97-bc45-750d84fea77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:27.085710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-3736d0df-bf0d-4d97-bc45-750d84fea77c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:27.086984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3736d0df-bf0d-4d97-bc45-750d84fea77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:27.098660 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7f885-4e59-44be-9d5c-1b4ba0772214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:27.099342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7f885-4e59-44be-9d5c-1b4ba0772214 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:27.099629 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7f885-4e59-44be-9d5c-1b4ba0772214 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:27.099906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7f885-4e59-44be-9d5c-1b4ba0772214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:27.214748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:27.239138 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7f885-4e59-44be-9d5c-1b4ba0772214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:27.240654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7f885-4e59-44be-9d5c-1b4ba0772214 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:27.245908 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 122/245] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:52:27.273474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736d0df-bf0d-4d97-bc45-750d84fea77c None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:27.305497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:27.371597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 124/246] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 14:52:27.378011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7f885-4e59-44be-9d5c-1b4ba0772214 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:27.399537 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736d0df-bf0d-4d97-bc45-750d84fea77c None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:27.444387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8501d-42e0-4b8d-9ca8-ce83321229fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:27.445274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8501d-42e0-4b8d-9ca8-ce83321229fd None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:27.445563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8501d-42e0-4b8d-9ca8-ce83321229fd None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:27.445957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8501d-42e0-4b8d-9ca8-ce83321229fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:27.460285 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8501d-42e0-4b8d-9ca8-ce83321229fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:27.469741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-16e8501d-42e0-4b8d-9ca8-ce83321229fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:27.469741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:27.469741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:27.469741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:27.469741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:27.469741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:27.469741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:27.469741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:27.469741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:27.469741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:27.469741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:27.469741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:27.469741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:27.469741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:27.469741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:27.469741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:27.470726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:27.470726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:27.470726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:27.470726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:27.470726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:27.470726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:27.470726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:27.470726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:27.470726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:27.470726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:27.470726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:27.470726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:27.470726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:27.470726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:27.470726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:27.470726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:27.471589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:27.471589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:27.471589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:27.471589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:27.471589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:27.471589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:27.471589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:27.471589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:27.471589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:27.471589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:27.471589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:27.471589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:27.474151 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 123/247] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:27.476442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:27.480060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8501d-42e0-4b8d-9ca8-ce83321229fd None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:27.492624 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7f885-4e59-44be-9d5c-1b4ba0772214 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:27.547034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150d865-d477-4797-b174-5fc51da2122f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:27.548271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150d865-d477-4797-b174-5fc51da2122f None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:27.548570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150d865-d477-4797-b174-5fc51da2122f None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:27.548855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150d865-d477-4797-b174-5fc51da2122f None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:27.550686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150d865-d477-4797-b174-5fc51da2122f None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:27.552190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150d865-d477-4797-b174-5fc51da2122f None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:27.554117 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 125/248] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:27 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:27.555248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46d778-9f05-427c-96f1-4863cc872ccb None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:27.555645 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46d778-9f05-427c-96f1-4863cc872ccb None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:27.555875 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46d778-9f05-427c-96f1-4863cc872ccb None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:27.557207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 126/249] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:27.565356 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46d778-9f05-427c-96f1-4863cc872ccb None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:27.607678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:27.633865 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8501d-42e0-4b8d-9ca8-ce83321229fd None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:27.680086 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920ca6f-9f2b-4b06-9ae8-f8f0434f89dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:27.681183 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920ca6f-9f2b-4b06-9ae8-f8f0434f89dc None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:27.681682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920ca6f-9f2b-4b06-9ae8-f8f0434f89dc None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:27.682176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920ca6f-9f2b-4b06-9ae8-f8f0434f89dc None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:27.683201 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920ca6f-9f2b-4b06-9ae8-f8f0434f89dc None admin] RBAC: Adding query filter params (name=member) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:27.683763 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920ca6f-9f2b-4b06-9ae8-f8f0434f89dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:27.685664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920ca6f-9f2b-4b06-9ae8-f8f0434f89dc None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:27.696766 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 124/250] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:27.698203 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eea77-55fd-44f9-beb8-d25bbd85cb4e None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:27.698682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eea77-55fd-44f9-beb8-d25bbd85cb4e None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:27.698908 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46d778-9f05-427c-96f1-4863cc872ccb None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:27.699204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eea77-55fd-44f9-beb8-d25bbd85cb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:27.719606 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-c06eea77-55fd-44f9-beb8-d25bbd85cb4e None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:27.721231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c06eea77-55fd-44f9-beb8-d25bbd85cb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:27.749259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba305e-b8f8-4793-9e74-a68273908d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:27.750248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba305e-b8f8-4793-9e74-a68273908d9c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:27.750908 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba305e-b8f8-4793-9e74-a68273908d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:27.751348 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba305e-b8f8-4793-9e74-a68273908d9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:27.752853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba305e-b8f8-4793-9e74-a68273908d9c None admin] RBAC: Adding query filter params (role.id=192149ed0eab4f039954e4617e7ee779, scope.project.id=cc9a9fe967674703a9776dca128bc387, user.id=464b5888738c43d1883e15171a05710c) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:27.753367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba305e-b8f8-4793-9e74-a68273908d9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:27.755722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba305e-b8f8-4793-9e74-a68273908d9c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:27.765696 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 127/251] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:27 2022] GET /identity/v3/role_assignments?user.id=464b5888738c43d1883e15171a05710c&scope.project.id=cc9a9fe967674703a9776dca128bc387&role.id=192149ed0eab4f039954e4617e7ee779 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:27.767012 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba305e-b8f8-4793-9e74-a68273908d9c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:27.799958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:27.818099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 125/252] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:27.859891 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06eea77-55fd-44f9-beb8-d25bbd85cb4e None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:27.942201 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba305e-b8f8-4793-9e74-a68273908d9c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:27.998879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba4e0b-34bf-4a20-bb35-5a860115086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:27.999816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba4e0b-34bf-4a20-bb35-5a860115086e None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:28.000226 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba4e0b-34bf-4a20-bb35-5a860115086e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:28.000607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba4e0b-34bf-4a20-bb35-5a860115086e None admin] PATH_INFO: `/v3/users/e0fa7d63fd914f3082862d1181b78157` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:28.004393 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba4e0b-34bf-4a20-bb35-5a860115086e None admin] RBAC: Authorizing `identity:get_user(user_id=e0fa7d63fd914f3082862d1181b78157)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:28.007315 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba4e0b-34bf-4a20-bb35-5a860115086e None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:28.020421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 128/253] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:27 2022] GET /identity/v3/users/e0fa7d63fd914f3082862d1181b78157 => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:28.026580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba4e0b-34bf-4a20-bb35-5a860115086e None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:28.128201 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:28.163525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:28.180947 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06eea77-55fd-44f9-beb8-d25bbd85cb4e None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:28.187822 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba4e0b-34bf-4a20-bb35-5a860115086e None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:28.252028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e942d-c0b5-4044-ab50-70f8573b17f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:28.253291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e942d-c0b5-4044-ab50-70f8573b17f8 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:28.253819 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e942d-c0b5-4044-ab50-70f8573b17f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:28.254704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e942d-c0b5-4044-ab50-70f8573b17f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:28.256570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5f4c4-7590-4fcc-be52-c711de1e1014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:28.257892 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5f4c4-7590-4fcc-be52-c711de1e1014 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:28.259729 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5f4c4-7590-4fcc-be52-c711de1e1014 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:28.261031 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5f4c4-7590-4fcc-be52-c711de1e1014 None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:28.263979 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f5f4c4-7590-4fcc-be52-c711de1e1014 None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:28.266942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f5f4c4-7590-4fcc-be52-c711de1e1014 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:28.269358 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 129/254] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:28 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:28.275581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5f4c4-7590-4fcc-be52-c711de1e1014 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:28.282251 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e942d-c0b5-4044-ab50-70f8573b17f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:28.291124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-c99e942d-c0b5-4044-ab50-70f8573b17f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:28.291124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:28.291124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:28.291124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:28.291124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:28.291124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:28.291124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:28.291124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:28.291124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:28.291124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:28.291124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:28.291124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:28.291124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:28.291124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:28.291124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:28.291124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:28.292239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:28.292239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:28.292239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:28.292239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:28.292239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:28.292239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:28.292239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:28.292239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:28.292239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:28.292239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:28.292239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:28.292239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:28.292239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:28.292239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:28.292239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:28.292239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:28.293298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:28.293298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:28.293298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:28.293298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:28.293298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:28.293298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:28.293298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:28.293298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:28.293298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:28.293298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:28.293298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:28.293298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:28.296641 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 126/255] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:52:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 438 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:52:28.308403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e942d-c0b5-4044-ab50-70f8573b17f8 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:28.490615 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5f4c4-7590-4fcc-be52-c711de1e1014 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:28.552729 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e942d-c0b5-4044-ab50-70f8573b17f8 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:28.578264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9e294-7735-48ff-a8ca-d90f3d0493fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:28.579215 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9e294-7735-48ff-a8ca-d90f3d0493fe None admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:28.579504 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9e294-7735-48ff-a8ca-d90f3d0493fe None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:28.579796 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9e294-7735-48ff-a8ca-d90f3d0493fe None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387/users/e0fa7d63fd914f3082862d1181b78157/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:28.584918 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9e294-7735-48ff-a8ca-d90f3d0493fe None admin] RBAC: Authorizing `identity:create_grant(project_id=cc9a9fe967674703a9776dca128bc387, user_id=e0fa7d63fd914f3082862d1181b78157, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:28.586426 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9e294-7735-48ff-a8ca-d90f3d0493fe None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:28.601211 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473f457-3fb0-489a-b0ad-83a57a8d0808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:28.602370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473f457-3fb0-489a-b0ad-83a57a8d0808 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:28.602946 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473f457-3fb0-489a-b0ad-83a57a8d0808 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:28.603462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473f457-3fb0-489a-b0ad-83a57a8d0808 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:28.604469 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473f457-3fb0-489a-b0ad-83a57a8d0808 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:28.605016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473f457-3fb0-489a-b0ad-83a57a8d0808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:28.607486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473f457-3fb0-489a-b0ad-83a57a8d0808 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:28.609850 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:28.619716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 127/256] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:28.621399 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354125e-c4b7-4793-b228-e5bf4948a65f None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:28.621805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354125e-c4b7-4793-b228-e5bf4948a65f None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:28.622193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354125e-c4b7-4793-b228-e5bf4948a65f None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:28.624614 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 128/257] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:28.629966 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354125e-c4b7-4793-b228-e5bf4948a65f None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:28.665459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 130/258] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:52:28 2022] PUT /identity/v3/projects/cc9a9fe967674703a9776dca128bc387/users/e0fa7d63fd914f3082862d1181b78157/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:52:28.667106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3321f3-bef0-41f0-9d56-e11e0fec0c8b None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:28.667557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3321f3-bef0-41f0-9d56-e11e0fec0c8b None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:28.668020 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3321f3-bef0-41f0-9d56-e11e0fec0c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:28.711753 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-8f3321f3-bef0-41f0-9d56-e11e0fec0c8b None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:28.713328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-8f3321f3-bef0-41f0-9d56-e11e0fec0c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:28.735822 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:28.756200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354125e-c4b7-4793-b228-e5bf4948a65f None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:28.799262 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3339056-6d8e-4fe2-9362-6938f0ed516d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:28.800365 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3339056-6d8e-4fe2-9362-6938f0ed516d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:28.800756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3339056-6d8e-4fe2-9362-6938f0ed516d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:28.801143 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3339056-6d8e-4fe2-9362-6938f0ed516d None admin] PATH_INFO: `/v3/users/22180e0b730a4c8187f67e3fb6275b0f` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:28.802524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3339056-6d8e-4fe2-9362-6938f0ed516d None admin] RBAC: Authorizing `identity:get_user(user_id=22180e0b730a4c8187f67e3fb6275b0f)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:28.804401 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3339056-6d8e-4fe2-9362-6938f0ed516d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:28.806440 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 129/259] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:28 2022] GET /identity/v3/users/22180e0b730a4c8187f67e3fb6275b0f => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:28.813317 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3339056-6d8e-4fe2-9362-6938f0ed516d None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:28.884673 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:28.926245 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 131/260] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:28.935570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3339056-6d8e-4fe2-9362-6938f0ed516d None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:28.962090 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3321f3-bef0-41f0-9d56-e11e0fec0c8b None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:29.010306 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae546b2-a1ba-4b0a-8098-806ee74bc1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:29.010903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae546b2-a1ba-4b0a-8098-806ee74bc1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:29.011191 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae546b2-a1ba-4b0a-8098-806ee74bc1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:29.011470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae546b2-a1ba-4b0a-8098-806ee74bc1f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:29.149820 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae546b2-a1ba-4b0a-8098-806ee74bc1f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:29.151351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae546b2-a1ba-4b0a-8098-806ee74bc1f4 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:29.175494 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3321f3-bef0-41f0-9d56-e11e0fec0c8b None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:29.246079 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffc300-1dd3-4396-bc32-e56134a17d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:29.252292 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffc300-1dd3-4396-bc32-e56134a17d1e None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:29.252292 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffc300-1dd3-4396-bc32-e56134a17d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:29.252292 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffc300-1dd3-4396-bc32-e56134a17d1e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:29.272546 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ffc300-1dd3-4396-bc32-e56134a17d1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:29.284701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-e7ffc300-1dd3-4396-bc32-e56134a17d1e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:29.284701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:29.284701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:29.284701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:29.284701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:29.284701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:29.284701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:29.284701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:29.284701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:29.284701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:29.284701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:29.284701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:29.284701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:29.284701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:29.284701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:29.284701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:29.286505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:29.286505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:29.286505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:29.286505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:29.286505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:29.286505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:29.286505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:29.286505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:29.286505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:29.286505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:29.286505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:29.286505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:29.286505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:29.286505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:29.286505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:29.286505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:29.288119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:29.288119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:29.288119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:29.288119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:29.288119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:29.288119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:29.288119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:29.288119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:29.288119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:29.288119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:29.288119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:29.288119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:29.291954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 132/261] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:52:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:52:29.296121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:29.298770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffc300-1dd3-4396-bc32-e56134a17d1e None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:29.320061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:29.367146 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 130/262] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:29.379599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae546b2-a1ba-4b0a-8098-806ee74bc1f4 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:29.537431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffc300-1dd3-4396-bc32-e56134a17d1e None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:29.608136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f87c87-8d7e-42c9-bd13-827d45278094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:29.608828 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f87c87-8d7e-42c9-bd13-827d45278094 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:29.609116 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f87c87-8d7e-42c9-bd13-827d45278094 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:29.609641 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f87c87-8d7e-42c9-bd13-827d45278094 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:29.610411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f87c87-8d7e-42c9-bd13-827d45278094 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:29.610750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f87c87-8d7e-42c9-bd13-827d45278094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:29.612719 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f87c87-8d7e-42c9-bd13-827d45278094 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:29.633015 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 133/263] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:52:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:29.634326 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae546b2-a1ba-4b0a-8098-806ee74bc1f4 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:29.641833 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f87c87-8d7e-42c9-bd13-827d45278094 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:29.700407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a8787-49ec-43b5-a196-d7876481c1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:29.702329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a8787-49ec-43b5-a196-d7876481c1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:29.702780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a8787-49ec-43b5-a196-d7876481c1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:29.703178 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a8787-49ec-43b5-a196-d7876481c1b1 None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:29.706422 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a8787-49ec-43b5-a196-d7876481c1b1 None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:29.708277 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a8787-49ec-43b5-a196-d7876481c1b1 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:29.710643 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 131/264] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:29 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:29.717177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a8787-49ec-43b5-a196-d7876481c1b1 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:29.769696 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:29.820117 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:29.908956 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f87c87-8d7e-42c9-bd13-827d45278094 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:29.930811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a8787-49ec-43b5-a196-d7876481c1b1 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:30.027298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9ce89-e4cf-4a6a-b795-a015c3254bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:30.027988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9ce89-e4cf-4a6a-b795-a015c3254bdd None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:30.028155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fd12a-89f7-4116-9a8f-6586c474501c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:30.028313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9ce89-e4cf-4a6a-b795-a015c3254bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:30.029165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9ce89-e4cf-4a6a-b795-a015c3254bdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:30.029908 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fd12a-89f7-4116-9a8f-6586c474501c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:30.030165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9ce89-e4cf-4a6a-b795-a015c3254bdd None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=cc9a9fe967674703a9776dca128bc387, user.id=22180e0b730a4c8187f67e3fb6275b0f) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:30.030320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fd12a-89f7-4116-9a8f-6586c474501c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:30.030454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fd12a-89f7-4116-9a8f-6586c474501c None admin] PATH_INFO: `/v3/users/22180e0b730a4c8187f67e3fb6275b0f` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:30.030616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9ce89-e4cf-4a6a-b795-a015c3254bdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:30.031755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787fd12a-89f7-4116-9a8f-6586c474501c None admin] RBAC: Authorizing `identity:get_user(user_id=22180e0b730a4c8187f67e3fb6275b0f)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:30.032019 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9ce89-e4cf-4a6a-b795-a015c3254bdd None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:30.033213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787fd12a-89f7-4116-9a8f-6586c474501c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:30.039976 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 132/265] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:29 2022] GET /identity/v3/role_assignments?user.id=22180e0b730a4c8187f67e3fb6275b0f&scope.project.id=cc9a9fe967674703a9776dca128bc387&role.id=5891920310b54787bd1152d736d2a3f1 => generated 608 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:30.043349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 134/266] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:29 2022] GET /identity/v3/users/22180e0b730a4c8187f67e3fb6275b0f => generated 283 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:30.049288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9ce89-e4cf-4a6a-b795-a015c3254bdd None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:30.199276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:30.208956 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9ce89-e4cf-4a6a-b795-a015c3254bdd None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:30.253783 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89087d9d-7f55-489c-8224-8036de43f7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:30.254464 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89087d9d-7f55-489c-8224-8036de43f7fa None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:30.254796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89087d9d-7f55-489c-8224-8036de43f7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:30.255090 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89087d9d-7f55-489c-8224-8036de43f7fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:30.343396 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89087d9d-7f55-489c-8224-8036de43f7fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:30.345034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89087d9d-7f55-489c-8224-8036de43f7fa None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:30.480230 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 133/267] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:30.481259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:30.486923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fd12a-89f7-4116-9a8f-6586c474501c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:30.655775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fd12a-89f7-4116-9a8f-6586c474501c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:30.697415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d9368-3450-41aa-a5e0-303162405bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:30.698297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d9368-3450-41aa-a5e0-303162405bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:30.698612 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d9368-3450-41aa-a5e0-303162405bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:30.698898 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d9368-3450-41aa-a5e0-303162405bf7 None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:30.700129 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d9368-3450-41aa-a5e0-303162405bf7 None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:30.701628 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d9368-3450-41aa-a5e0-303162405bf7 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:30.703733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 135/268] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:30 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:30.711668 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89087d9d-7f55-489c-8224-8036de43f7fa None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:30.864234 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89087d9d-7f55-489c-8224-8036de43f7fa None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:30.906528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f1ab4-22b5-4306-bc21-3c8969dd50fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:30.907229 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f1ab4-22b5-4306-bc21-3c8969dd50fe None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:30.907520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f1ab4-22b5-4306-bc21-3c8969dd50fe None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:30.907806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f1ab4-22b5-4306-bc21-3c8969dd50fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:30.908988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f1ab4-22b5-4306-bc21-3c8969dd50fe None admin] RBAC: Adding query filter params (role.id=a82e24db1e3c4f868a586e122dae393e, scope.project.id=cc9a9fe967674703a9776dca128bc387, user.id=22180e0b730a4c8187f67e3fb6275b0f) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:30.909152 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f1ab4-22b5-4306-bc21-3c8969dd50fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:30.910652 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f1ab4-22b5-4306-bc21-3c8969dd50fe None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:30.917660 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 134/269] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:30 2022] GET /identity/v3/role_assignments?user.id=22180e0b730a4c8187f67e3fb6275b0f&scope.project.id=cc9a9fe967674703a9776dca128bc387&role.id=a82e24db1e3c4f868a586e122dae393e => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:30.958328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7621c-dc04-4f2a-8703-025c1d17b5b4 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:30.958803 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7621c-dc04-4f2a-8703-025c1d17b5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:30.958959 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7621c-dc04-4f2a-8703-025c1d17b5b4 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:30.960354 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 136/270] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:30.978679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f7631-179e-404f-ab40-059a22129324 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:30.978982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f7631-179e-404f-ab40-059a22129324 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:30.979326 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f7631-179e-404f-ab40-059a22129324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:31.000627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-f87f7631-179e-404f-ab40-059a22129324 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:31.001377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5c4b4-2551-48f4-ba8a-e245d2dc225f None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:31.001668 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5c4b4-2551-48f4-ba8a-e245d2dc225f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:31.001946 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f87f7631-179e-404f-ab40-059a22129324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:31.002118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5c4b4-2551-48f4-ba8a-e245d2dc225f None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:31.003718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 137/271] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:31.010031 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e487c-37f7-4ee5-89e8-b7cf95054e56 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:31.010433 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e487c-37f7-4ee5-89e8-b7cf95054e56 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:31.010649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e487c-37f7-4ee5-89e8-b7cf95054e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:31.020957 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:31.035631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-225e487c-37f7-4ee5-89e8-b7cf95054e56 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:31.037028 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-225e487c-37f7-4ee5-89e8-b7cf95054e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:31.084553 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:31.096455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 135/272] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:31.098104 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5673047-1b7a-424f-bf61-67d6ea5d1ccf None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:31.098630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5673047-1b7a-424f-bf61-67d6ea5d1ccf None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:31.098984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5673047-1b7a-424f-bf61-67d6ea5d1ccf None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:31.100483 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 136/273] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:31.112137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686e211-e29f-405a-baf3-6ddc6bd9a8ce None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:31.112589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686e211-e29f-405a-baf3-6ddc6bd9a8ce None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:31.118676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686e211-e29f-405a-baf3-6ddc6bd9a8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:31.143489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-7686e211-e29f-405a-baf3-6ddc6bd9a8ce None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:31.144755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7686e211-e29f-405a-baf3-6ddc6bd9a8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:31.147373 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 138/274] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:52:31.148455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e487c-37f7-4ee5-89e8-b7cf95054e56 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:31.258974 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 137/275] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:31.259839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686e211-e29f-405a-baf3-6ddc6bd9a8ce None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:31.320962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e487c-37f7-4ee5-89e8-b7cf95054e56 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:31.360391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bf5f4-ef02-4c6b-9ced-e8bc1bc606f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:31.361200 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bf5f4-ef02-4c6b-9ced-e8bc1bc606f9 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:31.361487 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bf5f4-ef02-4c6b-9ced-e8bc1bc606f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:31.361766 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bf5f4-ef02-4c6b-9ced-e8bc1bc606f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:31.363001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bf5f4-ef02-4c6b-9ced-e8bc1bc606f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:31.364499 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bf5f4-ef02-4c6b-9ced-e8bc1bc606f9 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:31.366326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 139/276] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:52:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:31.371519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bf5f4-ef02-4c6b-9ced-e8bc1bc606f9 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:31.375023 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:31.391395 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686e211-e29f-405a-baf3-6ddc6bd9a8ce None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:31.414178 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:31.429176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae4ad8-a807-4a49-ae86-52ff1cb9991e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:31.430151 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae4ad8-a807-4a49-ae86-52ff1cb9991e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:31.430437 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae4ad8-a807-4a49-ae86-52ff1cb9991e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:31.430748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae4ad8-a807-4a49-ae86-52ff1cb9991e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:31.442444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae4ad8-a807-4a49-ae86-52ff1cb9991e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:31.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-56ae4ad8-a807-4a49-ae86-52ff1cb9991e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:31.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:31.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:31.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:31.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:31.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:31.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:31.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:31.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:31.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:31.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:31.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:31.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:31.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:31.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:31.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:31.452451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:31.452451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:31.452451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:31.452451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:31.452451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:31.452451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:31.452451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:31.452451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:31.452451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:31.452451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:31.452451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:31.452451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:31.452451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:31.452451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:31.452451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:31.452451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:31.453312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:31.453312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:31.453312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:31.453312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:31.453312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:31.453312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:31.453312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:31.453312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:31.453312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:31.453312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:31.453312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:31.453312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:31.456297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 138/277] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:31.457540 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae4ad8-a807-4a49-ae86-52ff1cb9991e None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:31.492902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bf5f4-ef02-4c6b-9ced-e8bc1bc606f9 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:31.531049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa865a9-d9d4-4b06-932a-73db5d3bd779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:31.532002 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa865a9-d9d4-4b06-932a-73db5d3bd779 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:31.532293 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa865a9-d9d4-4b06-932a-73db5d3bd779 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:31.532572 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa865a9-d9d4-4b06-932a-73db5d3bd779 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:31.544540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa865a9-d9d4-4b06-932a-73db5d3bd779 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:31.553381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-4aa865a9-d9d4-4b06-932a-73db5d3bd779 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:31.553381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:31.553381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:31.553381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:31.553381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:31.553381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:31.553381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:31.553381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:31.553381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:31.553381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:31.553381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:31.553381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:31.553381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:31.553381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:31.553381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:31.553381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:31.554505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:31.554505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:31.554505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:31.554505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:31.554505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:31.554505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:31.554505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:31.554505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:31.554505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:31.554505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:31.554505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:31.554505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:31.554505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:31.554505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:31.554505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:31.554505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:31.555545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:31.555545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:31.555545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:31.555545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:31.555545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:31.555545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:31.555545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:31.555545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:31.555545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:31.555545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:31.555545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:31.555545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:31.555545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 140/278] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:31.560594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa865a9-d9d4-4b06-932a-73db5d3bd779 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:31.576739 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae4ad8-a807-4a49-ae86-52ff1cb9991e None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:31.615953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cb0f3-5c64-4fad-94a8-f0985116f4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:31.616935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cb0f3-5c64-4fad-94a8-f0985116f4ed None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:31.617390 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cb0f3-5c64-4fad-94a8-f0985116f4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:31.617669 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cb0f3-5c64-4fad-94a8-f0985116f4ed None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:31.618388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485cb0f3-5c64-4fad-94a8-f0985116f4ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:31.619910 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485cb0f3-5c64-4fad-94a8-f0985116f4ed None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:31.636163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:31.677024 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa865a9-d9d4-4b06-932a-73db5d3bd779 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:31.711195 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda460a9-37fd-49be-b958-87f91989b998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:31.711872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda460a9-37fd-49be-b958-87f91989b998 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:31.712158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda460a9-37fd-49be-b958-87f91989b998 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:31.712434 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda460a9-37fd-49be-b958-87f91989b998 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:31.713188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda460a9-37fd-49be-b958-87f91989b998 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:31.713523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda460a9-37fd-49be-b958-87f91989b998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:31.714967 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda460a9-37fd-49be-b958-87f91989b998 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:31.715689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 139/279] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:52:31 2022] POST /identity/v3/users => generated 293 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:52:31.717162 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cb0f3-5c64-4fad-94a8-f0985116f4ed None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:31.725848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 141/280] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:31.727620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbce42-95e8-4108-9b38-fa1014510069 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:31.727926 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbce42-95e8-4108-9b38-fa1014510069 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:31.728191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbce42-95e8-4108-9b38-fa1014510069 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:31.729586 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 142/281] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:31.732155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbce42-95e8-4108-9b38-fa1014510069 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:31.773453 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:31.855751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cb0f3-5c64-4fad-94a8-f0985116f4ed None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:31.856011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbce42-95e8-4108-9b38-fa1014510069 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:31.887186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:31.900496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358a663-c8a3-40aa-9cf8-2b46125e7904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:31.901645 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84762a-6641-4367-bc43-988d0ce7c680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:31.901774 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358a663-c8a3-40aa-9cf8-2b46125e7904 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:31.901852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84762a-6641-4367-bc43-988d0ce7c680 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:31.901922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358a663-c8a3-40aa-9cf8-2b46125e7904 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:31.901989 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84762a-6641-4367-bc43-988d0ce7c680 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:31.902069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358a663-c8a3-40aa-9cf8-2b46125e7904 None admin] PATH_INFO: `/v3/users/464b5888738c43d1883e15171a05710c` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:31.902159 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84762a-6641-4367-bc43-988d0ce7c680 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:31.903989 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84762a-6641-4367-bc43-988d0ce7c680 None admin] RBAC: Adding query filter params (name=member) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:31.903989 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84762a-6641-4367-bc43-988d0ce7c680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:31.904157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0358a663-c8a3-40aa-9cf8-2b46125e7904 None admin] RBAC: Authorizing `identity:get_user(user_id=464b5888738c43d1883e15171a05710c)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:31.905860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84762a-6641-4367-bc43-988d0ce7c680 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:31.905975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0358a663-c8a3-40aa-9cf8-2b46125e7904 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:31.907861 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 143/282] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:31 2022] GET /identity/v3/users/464b5888738c43d1883e15171a05710c => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:31.909519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c1acd-9a81-441d-8e14-fc4a7951ac39 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:31.909519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c1acd-9a81-441d-8e14-fc4a7951ac39 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:31.909519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c1acd-9a81-441d-8e14-fc4a7951ac39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:31.920631 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 140/283] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:31.924541 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84762a-6641-4367-bc43-988d0ce7c680 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:31.929343 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-e14c1acd-9a81-441d-8e14-fc4a7951ac39 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:31.930670 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e14c1acd-9a81-441d-8e14-fc4a7951ac39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:32.018701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 144/284] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:32.019569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c1acd-9a81-441d-8e14-fc4a7951ac39 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:32.074848 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:32.080488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84762a-6641-4367-bc43-988d0ce7c680 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:32.133312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38657b-c246-4dbe-9fa8-a04525e739e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:32.134195 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38657b-c246-4dbe-9fa8-a04525e739e5 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:32.134480 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38657b-c246-4dbe-9fa8-a04525e739e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:32.134800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38657b-c246-4dbe-9fa8-a04525e739e5 None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:32.136031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38657b-c246-4dbe-9fa8-a04525e739e5 None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:32.137607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38657b-c246-4dbe-9fa8-a04525e739e5 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:32.139435 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 141/285] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:31 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:32.140207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38657b-c246-4dbe-9fa8-a04525e739e5 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:32.163433 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c1acd-9a81-441d-8e14-fc4a7951ac39 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:32.194059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:32.207915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d3bea-8c34-4e55-8595-e6a9f1f87e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:32.214620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d3bea-8c34-4e55-8595-e6a9f1f87e36 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:32.214620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d3bea-8c34-4e55-8595-e6a9f1f87e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:32.214620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d3bea-8c34-4e55-8595-e6a9f1f87e36 None admin] PATH_INFO: `/v3/users/750f4772f93b4df080f4cd9bbf03a3fb` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:32.225588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d3bea-8c34-4e55-8595-e6a9f1f87e36 None admin] RBAC: Authorizing `identity:get_user(user_id=750f4772f93b4df080f4cd9bbf03a3fb)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:32.227206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d3bea-8c34-4e55-8595-e6a9f1f87e36 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:32.229426 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 145/286] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:32 2022] GET /identity/v3/users/750f4772f93b4df080f4cd9bbf03a3fb => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:32.236130 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d3bea-8c34-4e55-8595-e6a9f1f87e36 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:32.256950 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38657b-c246-4dbe-9fa8-a04525e739e5 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:32.303581 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa3806-acc1-4ed9-93bc-8ea8c2b12595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:32.304398 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa3806-acc1-4ed9-93bc-8ea8c2b12595 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:32.304706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa3806-acc1-4ed9-93bc-8ea8c2b12595 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:32.304980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa3806-acc1-4ed9-93bc-8ea8c2b12595 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:32.316457 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa3806-acc1-4ed9-93bc-8ea8c2b12595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:32.324441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-69fa3806-acc1-4ed9-93bc-8ea8c2b12595 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:32.324441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:32.324441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:32.324441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:32.324441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:32.324441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:32.324441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:32.324441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:32.324441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:32.324441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:32.324441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:32.324441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:32.324441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:32.324441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:32.324441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:32.324441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:32.325959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:32.325959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:32.325959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:32.325959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:32.325959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:32.325959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:32.325959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:32.325959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:32.325959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:32.325959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:32.325959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:32.325959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:32.325959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:32.325959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:32.325959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:32.325959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:32.326786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:32.326786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:32.326786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:32.326786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:32.326786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:32.326786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:32.326786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:32.326786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:32.326786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:32.326786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:32.326786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:32.326786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:32.326786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 142/287] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:32.328005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa3806-acc1-4ed9-93bc-8ea8c2b12595 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:32.357902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:32.360763 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d3bea-8c34-4e55-8595-e6a9f1f87e36 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:32.400334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b641a-a9ff-43d7-90c0-26f1d89f84c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:32.401589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b641a-a9ff-43d7-90c0-26f1d89f84c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:32.402410 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b641a-a9ff-43d7-90c0-26f1d89f84c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:32.402872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b641a-a9ff-43d7-90c0-26f1d89f84c5 None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387/users/464b5888738c43d1883e15171a05710c/roles/192149ed0eab4f039954e4617e7ee779` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:32.405366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b641a-a9ff-43d7-90c0-26f1d89f84c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc9a9fe967674703a9776dca128bc387, user_id=464b5888738c43d1883e15171a05710c, role_id=192149ed0eab4f039954e4617e7ee779)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:32.407177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b641a-a9ff-43d7-90c0-26f1d89f84c5 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:32.445572 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa3806-acc1-4ed9-93bc-8ea8c2b12595 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:32.483891 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 146/288] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:52:32 2022] PUT /identity/v3/projects/cc9a9fe967674703a9776dca128bc387/users/464b5888738c43d1883e15171a05710c/roles/192149ed0eab4f039954e4617e7ee779 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:52:32.485499 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b641a-a9ff-43d7-90c0-26f1d89f84c5 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:32.487326 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fcf7c-5526-4da0-bb94-df6e33295bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:32.488068 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fcf7c-5526-4da0-bb94-df6e33295bac None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:32.488358 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fcf7c-5526-4da0-bb94-df6e33295bac None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:32.488640 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fcf7c-5526-4da0-bb94-df6e33295bac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:32.489563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4fcf7c-5526-4da0-bb94-df6e33295bac None admin] RBAC: Adding query filter params (role.id=3ab0ee91b3284a65a0dd5696a6250883, user.id=750f4772f93b4df080f4cd9bbf03a3fb, scope.system=all) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:32.490142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4fcf7c-5526-4da0-bb94-df6e33295bac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:32.492204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4fcf7c-5526-4da0-bb94-df6e33295bac None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:32.506580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 143/289] 10.222.0.83 () {56 vars in 1308 bytes} [Mon Jul 18 14:52:32 2022] GET /identity/v3/role_assignments?user.id=750f4772f93b4df080f4cd9bbf03a3fb&scope.system=all&role.id=3ab0ee91b3284a65a0dd5696a6250883 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:32.556998 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:32.632447 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b641a-a9ff-43d7-90c0-26f1d89f84c5 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:32.675574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eaa5c0-9a61-4460-8b56-3776cdc6a4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:32.676171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eaa5c0-9a61-4460-8b56-3776cdc6a4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:32.676458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eaa5c0-9a61-4460-8b56-3776cdc6a4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:32.676740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eaa5c0-9a61-4460-8b56-3776cdc6a4f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:32.677520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eaa5c0-9a61-4460-8b56-3776cdc6a4f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:32.677830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eaa5c0-9a61-4460-8b56-3776cdc6a4f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:32.679299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eaa5c0-9a61-4460-8b56-3776cdc6a4f6 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:32.691558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 147/290] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:32.696949 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fcf7c-5526-4da0-bb94-df6e33295bac None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:32.845841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:32.857975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fcf7c-5526-4da0-bb94-df6e33295bac None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:32.901440 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12881bb-5872-4f2c-b129-adce08b44a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:32.902432 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12881bb-5872-4f2c-b129-adce08b44a35 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:32.902752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12881bb-5872-4f2c-b129-adce08b44a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:32.903027 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12881bb-5872-4f2c-b129-adce08b44a35 None admin] PATH_INFO: `/v3/users/e0fa7d63fd914f3082862d1181b78157` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:32.904603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12881bb-5872-4f2c-b129-adce08b44a35 None admin] RBAC: Authorizing `identity:get_user(user_id=e0fa7d63fd914f3082862d1181b78157)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:32.906144 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12881bb-5872-4f2c-b129-adce08b44a35 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:32.907985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 144/291] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:32 2022] GET /identity/v3/users/e0fa7d63fd914f3082862d1181b78157 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:32.913942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eaa5c0-9a61-4460-8b56-3776cdc6a4f6 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:33.075369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eaa5c0-9a61-4460-8b56-3776cdc6a4f6 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:33.077098 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:33.128294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675bdc6-ab21-4bbc-a846-189685ad9dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:33.129013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675bdc6-ab21-4bbc-a846-189685ad9dbe None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:33.129308 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675bdc6-ab21-4bbc-a846-189685ad9dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:33.129598 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675bdc6-ab21-4bbc-a846-189685ad9dbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:33.244454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675bdc6-ab21-4bbc-a846-189685ad9dbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:33.246309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675bdc6-ab21-4bbc-a846-189685ad9dbe None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:33.412272 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 148/292] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 14:52:33.418020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12881bb-5872-4f2c-b129-adce08b44a35 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:33.563620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12881bb-5872-4f2c-b129-adce08b44a35 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:33.624424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc622b5-cd24-417b-b763-8252e9572e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:33.625364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc622b5-cd24-417b-b763-8252e9572e0f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:33.625647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc622b5-cd24-417b-b763-8252e9572e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:33.625923 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc622b5-cd24-417b-b763-8252e9572e0f None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:33.627301 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc622b5-cd24-417b-b763-8252e9572e0f None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:33.628767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc622b5-cd24-417b-b763-8252e9572e0f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:33.630794 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 145/293] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:33 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:33.636577 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675bdc6-ab21-4bbc-a846-189685ad9dbe None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:33.642563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:33.790386 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675bdc6-ab21-4bbc-a846-189685ad9dbe None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:33.840685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4ea42-b428-45d3-b3a1-6cac17fc5600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:33.841734 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4ea42-b428-45d3-b3a1-6cac17fc5600 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:33.842263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4ea42-b428-45d3-b3a1-6cac17fc5600 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:33.842788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4ea42-b428-45d3-b3a1-6cac17fc5600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:33.843885 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4ea42-b428-45d3-b3a1-6cac17fc5600 None admin] RBAC: Adding query filter params (role.id=3ab0ee91b3284a65a0dd5696a6250883, scope.project.id=cc9a9fe967674703a9776dca128bc387, user.id=e0fa7d63fd914f3082862d1181b78157) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:33.844419 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4ea42-b428-45d3-b3a1-6cac17fc5600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:33.846156 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4ea42-b428-45d3-b3a1-6cac17fc5600 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:33.860864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 149/294] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:33 2022] GET /identity/v3/role_assignments?user.id=e0fa7d63fd914f3082862d1181b78157&scope.project.id=cc9a9fe967674703a9776dca128bc387&role.id=3ab0ee91b3284a65a0dd5696a6250883 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:34.186702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b15ffc-7768-40de-978b-1c89b2ff6bde None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:34.186998 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b15ffc-7768-40de-978b-1c89b2ff6bde None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:34.187302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b15ffc-7768-40de-978b-1c89b2ff6bde None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:34.188617 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 146/295] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:34.194495 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee6040-9dc0-4ec0-896f-bb2b9eaca101 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:34.195214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee6040-9dc0-4ec0-896f-bb2b9eaca101 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:34.195754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee6040-9dc0-4ec0-896f-bb2b9eaca101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:34.217036 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-04ee6040-9dc0-4ec0-896f-bb2b9eaca101 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:34.218706 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-04ee6040-9dc0-4ec0-896f-bb2b9eaca101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:34.291557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:34.299967 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 150/296] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:34.325808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b15ffc-7768-40de-978b-1c89b2ff6bde None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:34.359502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:34.527826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b15ffc-7768-40de-978b-1c89b2ff6bde None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:34.569377 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce93567-7f5f-49fc-91b1-f6e236e88855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:34.570182 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce93567-7f5f-49fc-91b1-f6e236e88855 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:34.570467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce93567-7f5f-49fc-91b1-f6e236e88855 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:34.570773 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce93567-7f5f-49fc-91b1-f6e236e88855 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:34.572179 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce93567-7f5f-49fc-91b1-f6e236e88855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:34.572584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b91ed1-16c7-49ce-be77-332a560487e0 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:34.572883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b91ed1-16c7-49ce-be77-332a560487e0 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:34.573174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b91ed1-16c7-49ce-be77-332a560487e0 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:34.573652 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce93567-7f5f-49fc-91b1-f6e236e88855 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:34.574900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 151/297] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:34.577109 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 147/298] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:52:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:34.582938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce93567-7f5f-49fc-91b1-f6e236e88855 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:34.583564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e25fc-bd23-4144-8b58-89ece758981c None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:34.584060 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e25fc-bd23-4144-8b58-89ece758981c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:34.584447 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e25fc-bd23-4144-8b58-89ece758981c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:34.602798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-468e25fc-bd23-4144-8b58-89ece758981c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:34.604190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-468e25fc-bd23-4144-8b58-89ece758981c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:34.680686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 152/299] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:34.705674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e25fc-bd23-4144-8b58-89ece758981c None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:34.708221 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:34.715925 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce93567-7f5f-49fc-91b1-f6e236e88855 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:34.753696 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9994e-db55-490e-ba62-7947539249b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:34.754568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9994e-db55-490e-ba62-7947539249b2 None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:34.754972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9994e-db55-490e-ba62-7947539249b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:34.755360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9994e-db55-490e-ba62-7947539249b2 None admin] PATH_INFO: `/v3/groups` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:34.756160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9994e-db55-490e-ba62-7947539249b2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:34.757810 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9994e-db55-490e-ba62-7947539249b2 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:34.821674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e25fc-bd23-4144-8b58-89ece758981c None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:34.850808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 148/300] 10.222.0.83 () {60 vars in 1134 bytes} [Mon Jul 18 14:52:34 2022] POST /identity/v3/groups => generated 229 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:52:34.868247 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:34.877244 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599772e-f438-4c47-b1dc-030f31262f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:34.878153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599772e-f438-4c47-b1dc-030f31262f80 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:34.878449 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599772e-f438-4c47-b1dc-030f31262f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:34.878816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599772e-f438-4c47-b1dc-030f31262f80 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:34.892501 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599772e-f438-4c47-b1dc-030f31262f80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:34.901101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-2599772e-f438-4c47-b1dc-030f31262f80 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:34.901101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:34.901101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:34.901101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:34.901101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:34.901101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:34.901101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:34.901101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:34.901101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:34.901101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:34.901101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:34.901101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:34.901101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:34.901101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:34.901101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:34.901101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:34.902332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:34.902332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:34.902332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:34.902332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:34.902332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:34.902332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:34.902332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:34.902332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:34.902332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:34.902332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:34.902332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:34.902332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:34.902332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:34.902332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:34.902332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:34.902332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:34.903370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:34.903370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:34.903370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:34.903370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:34.903370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:34.903370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:34.903370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:34.903370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:34.903370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:34.903370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:34.903370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:34.903370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:34.903370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 153/301] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:52:34.915734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9994e-db55-490e-ba62-7947539249b2 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:35.050695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9994e-db55-490e-ba62-7947539249b2 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:35.089725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d1890-a750-470a-bbc6-685db8d5fbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:35.090390 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d1890-a750-470a-bbc6-685db8d5fbc5 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:35.091200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d1890-a750-470a-bbc6-685db8d5fbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:35.091484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d1890-a750-470a-bbc6-685db8d5fbc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:35.092328 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d1890-a750-470a-bbc6-685db8d5fbc5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:35.093082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d1890-a750-470a-bbc6-685db8d5fbc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:35.094100 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d1890-a750-470a-bbc6-685db8d5fbc5 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:35.103610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 149/302] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:35.110245 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599772e-f438-4c47-b1dc-030f31262f80 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:35.178205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5426c7-15e4-44c8-8a9f-d5fe48898f0d None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:35.178859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5426c7-15e4-44c8-8a9f-d5fe48898f0d None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:35.179415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5426c7-15e4-44c8-8a9f-d5fe48898f0d None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:35.181325 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 150/303] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:35.188101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f1287-4a01-485b-9cf6-c1d9261ec3da None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:35.188744 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f1287-4a01-485b-9cf6-c1d9261ec3da None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:35.189236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f1287-4a01-485b-9cf6-c1d9261ec3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:35.208173 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-400f1287-4a01-485b-9cf6-c1d9261ec3da None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:35.209816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-400f1287-4a01-485b-9cf6-c1d9261ec3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:35.224133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:35.246686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599772e-f438-4c47-b1dc-030f31262f80 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:35.286923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f8436-2d02-43bf-b2da-56d2047d015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:35.288029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f8436-2d02-43bf-b2da-56d2047d015f None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:35.288177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f8436-2d02-43bf-b2da-56d2047d015f None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:35.288407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f8436-2d02-43bf-b2da-56d2047d015f None admin] PATH_INFO: `/v3/users/03ef64d794a14609b9516d48fd4a962c` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:35.296603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 151/304] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:35.300008 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dbff4-00de-49d2-8045-e8d8afe54270 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:35.300295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dbff4-00de-49d2-8045-e8d8afe54270 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:35.300573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dbff4-00de-49d2-8045-e8d8afe54270 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:35.302121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 152/305] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:35.307766 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194904a-b2b4-4acf-9c9a-986dca726620 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:35.308359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194904a-b2b4-4acf-9c9a-986dca726620 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:35.308359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194904a-b2b4-4acf-9c9a-986dca726620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:35.318848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f8436-2d02-43bf-b2da-56d2047d015f None admin] RBAC: Authorizing `identity:get_user(user_id=03ef64d794a14609b9516d48fd4a962c)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:35.318848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f8436-2d02-43bf-b2da-56d2047d015f None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:35.318848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 154/306] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:35 2022] GET /identity/v3/users/03ef64d794a14609b9516d48fd4a962c => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:35.320976 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f8436-2d02-43bf-b2da-56d2047d015f None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:35.328270 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-e194904a-b2b4-4acf-9c9a-986dca726620 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:35.329542 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e194904a-b2b4-4acf-9c9a-986dca726620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:35.377915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:35.415695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 153/307] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:52:35.420137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194904a-b2b4-4acf-9c9a-986dca726620 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:35.443896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f8436-2d02-43bf-b2da-56d2047d015f None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:35.488733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9ab1e-bad6-4d49-82ce-a2b217cff3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:35.489413 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9ab1e-bad6-4d49-82ce-a2b217cff3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:35.489722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9ab1e-bad6-4d49-82ce-a2b217cff3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:35.490005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9ab1e-bad6-4d49-82ce-a2b217cff3d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:35.490916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9ab1e-bad6-4d49-82ce-a2b217cff3d5 None admin] RBAC: Adding query filter params (role.id=192149ed0eab4f039954e4617e7ee779, user.id=03ef64d794a14609b9516d48fd4a962c, scope.system=all) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:35.491231 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9ab1e-bad6-4d49-82ce-a2b217cff3d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:35.492754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9ab1e-bad6-4d49-82ce-a2b217cff3d5 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:35.498454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 155/308] 10.222.0.83 () {56 vars in 1308 bytes} [Mon Jul 18 14:52:35 2022] GET /identity/v3/role_assignments?user.id=03ef64d794a14609b9516d48fd4a962c&scope.system=all&role.id=192149ed0eab4f039954e4617e7ee779 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:35.499240 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9ab1e-bad6-4d49-82ce-a2b217cff3d5 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:35.529888 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:35.559154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194904a-b2b4-4acf-9c9a-986dca726620 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:35.614530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494c5e8-abd6-4f8c-9065-700967489492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:35.615408 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494c5e8-abd6-4f8c-9065-700967489492 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:35.615692 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494c5e8-abd6-4f8c-9065-700967489492 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:35.615963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494c5e8-abd6-4f8c-9065-700967489492 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:35.632843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f494c5e8-abd6-4f8c-9065-700967489492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:35.645035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-f494c5e8-abd6-4f8c-9065-700967489492 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:35.645035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:35.645035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:35.645035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:35.645035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:35.645035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:35.645035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:35.645035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:35.645035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:35.645035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:35.645035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:35.645035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:35.645035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:35.645035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:35.645035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:35.645035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:35.646077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:35.646077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:35.646077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:35.646077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:35.646077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:35.646077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:35.646077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:35.646077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:35.646077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:35.646077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:35.646077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:35.646077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:35.646077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:35.646077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:35.646077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:35.646077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:35.646988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:35.646988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:35.646988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:35.646988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:35.646988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:35.646988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:35.646988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:35.646988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:35.646988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:35.646988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:35.646988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:35.646988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:35.646988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 154/309] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:35.651888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9ab1e-bad6-4d49-82ce-a2b217cff3d5 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:35.655897 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494c5e8-abd6-4f8c-9065-700967489492 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:35.664422 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:35.702352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2fac8-63f4-4d61-800e-1154064b1a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:35.703289 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2fac8-63f4-4d61-800e-1154064b1a23 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:35.703513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2fac8-63f4-4d61-800e-1154064b1a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:35.703806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2fac8-63f4-4d61-800e-1154064b1a23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:35.726714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2fac8-63f4-4d61-800e-1154064b1a23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:35.738259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-e6f2fac8-63f4-4d61-800e-1154064b1a23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:35.738259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:35.738259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:35.738259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:35.738259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:35.738259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:35.738259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:35.738259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:35.738259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:35.738259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:35.738259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:35.738259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:35.738259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:35.738259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:35.738259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:35.738259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:35.739299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:35.739299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:35.739299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:35.739299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:35.739299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:35.739299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:35.739299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:35.739299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:35.739299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:35.739299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:35.739299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:35.739299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:35.739299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:35.739299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:35.739299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:35.739299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:35.740118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:35.740118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:35.740118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:35.740118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:35.740118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:35.740118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:35.740118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:35.740118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:35.740118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:35.740118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:35.740118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:35.740118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:35.740743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 156/310] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:35.746368 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2fac8-63f4-4d61-800e-1154064b1a23 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:35.802754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494c5e8-abd6-4f8c-9065-700967489492 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:35.843285 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82083cbd-63bd-4c52-bc8e-b29383ca0144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:35.844147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82083cbd-63bd-4c52-bc8e-b29383ca0144 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:35.844559 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82083cbd-63bd-4c52-bc8e-b29383ca0144 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:35.845047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82083cbd-63bd-4c52-bc8e-b29383ca0144 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:35.846022 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82083cbd-63bd-4c52-bc8e-b29383ca0144 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:35.846436 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82083cbd-63bd-4c52-bc8e-b29383ca0144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:35.848013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82083cbd-63bd-4c52-bc8e-b29383ca0144 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:35.850994 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:35.855430 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2fac8-63f4-4d61-800e-1154064b1a23 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:35.860510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 155/311] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:35.872603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82083cbd-63bd-4c52-bc8e-b29383ca0144 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:35.893379 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645cabc-c20f-4708-815e-5485f72480dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:35.894018 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645cabc-c20f-4708-815e-5485f72480dc None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:35.894311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645cabc-c20f-4708-815e-5485f72480dc None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:35.894648 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645cabc-c20f-4708-815e-5485f72480dc None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:35.895462 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645cabc-c20f-4708-815e-5485f72480dc None admin] RBAC: Adding query filter params (name=member) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:35.895796 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645cabc-c20f-4708-815e-5485f72480dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:35.897193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645cabc-c20f-4708-815e-5485f72480dc None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:35.914236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 157/312] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:35.915764 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645cabc-c20f-4708-815e-5485f72480dc None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:35.957095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:36.023998 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82083cbd-63bd-4c52-bc8e-b29383ca0144 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:36.067493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645cabc-c20f-4708-815e-5485f72480dc None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:36.101837 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4b544-af64-4f95-b8ac-202c7610c59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:36.104220 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4b544-af64-4f95-b8ac-202c7610c59a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:36.105164 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4b544-af64-4f95-b8ac-202c7610c59a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:36.105957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4b544-af64-4f95-b8ac-202c7610c59a None admin] PATH_INFO: `/v3/users/464b5888738c43d1883e15171a05710c` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:36.110095 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4b544-af64-4f95-b8ac-202c7610c59a None admin] RBAC: Authorizing `identity:get_user(user_id=464b5888738c43d1883e15171a05710c)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:36.113510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4b544-af64-4f95-b8ac-202c7610c59a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:36.116008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812d80c-7dda-4c2f-85ab-84e7a49d6a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:36.117024 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812d80c-7dda-4c2f-85ab-84e7a49d6a71 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:36.117317 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812d80c-7dda-4c2f-85ab-84e7a49d6a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:36.117573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812d80c-7dda-4c2f-85ab-84e7a49d6a71 None admin] PATH_INFO: `/v3/users/750f4772f93b4df080f4cd9bbf03a3fb` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:36.118881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 156/313] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:35 2022] GET /identity/v3/users/464b5888738c43d1883e15171a05710c => generated 297 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:36.119244 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812d80c-7dda-4c2f-85ab-84e7a49d6a71 None admin] RBAC: Authorizing `identity:get_user(user_id=750f4772f93b4df080f4cd9bbf03a3fb)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:36.120566 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812d80c-7dda-4c2f-85ab-84e7a49d6a71 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:36.126605 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 158/314] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:35 2022] GET /identity/v3/users/750f4772f93b4df080f4cd9bbf03a3fb => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:36.127289 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4b544-af64-4f95-b8ac-202c7610c59a None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:36.138715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812d80c-7dda-4c2f-85ab-84e7a49d6a71 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:36.282980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:36.335120 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4b544-af64-4f95-b8ac-202c7610c59a None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:36.337171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:36.345706 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812d80c-7dda-4c2f-85ab-84e7a49d6a71 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:36.406388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f8d00-0d15-490c-bdd9-43447a4e091b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:36.407490 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f8d00-0d15-490c-bdd9-43447a4e091b None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:36.407982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f8d00-0d15-490c-bdd9-43447a4e091b None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:36.408475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f8d00-0d15-490c-bdd9-43447a4e091b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:36.416213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db986af1-2970-4a32-a7f9-d7206023721b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:36.418004 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db986af1-2970-4a32-a7f9-d7206023721b None admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:36.418853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db986af1-2970-4a32-a7f9-d7206023721b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:36.419614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db986af1-2970-4a32-a7f9-d7206023721b None admin] PATH_INFO: `/v3/system/users/750f4772f93b4df080f4cd9bbf03a3fb/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:36.422370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db986af1-2970-4a32-a7f9-d7206023721b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=750f4772f93b4df080f4cd9bbf03a3fb, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:36.425121 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db986af1-2970-4a32-a7f9-d7206023721b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:36.493426 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f8d00-0d15-490c-bdd9-43447a4e091b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:36.494958 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f8d00-0d15-490c-bdd9-43447a4e091b None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:36.527344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 159/315] 10.222.0.83 () {58 vars in 1334 bytes} [Mon Jul 18 14:52:36 2022] PUT /identity/v3/system/users/750f4772f93b4df080f4cd9bbf03a3fb/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:52:36.568831 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:36.612697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 157/316] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:36.618701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db986af1-2970-4a32-a7f9-d7206023721b None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:36.745058 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db986af1-2970-4a32-a7f9-d7206023721b None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:36.787169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233c633-590c-4109-9ce7-401daac20e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:36.788045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233c633-590c-4109-9ce7-401daac20e33 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:36.788333 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233c633-590c-4109-9ce7-401daac20e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:36.788612 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233c633-590c-4109-9ce7-401daac20e33 None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:36.789836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233c633-590c-4109-9ce7-401daac20e33 None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:36.791396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233c633-590c-4109-9ce7-401daac20e33 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:36.793434 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 160/317] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:36 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:36.800077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f8d00-0d15-490c-bdd9-43447a4e091b None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:36.909125 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f8d00-0d15-490c-bdd9-43447a4e091b None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:36.946919 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8f9cb-aa1e-49b0-b8f6-36841de1a52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:36.947586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8f9cb-aa1e-49b0-b8f6-36841de1a52f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:36.947922 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8f9cb-aa1e-49b0-b8f6-36841de1a52f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:36.948208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8f9cb-aa1e-49b0-b8f6-36841de1a52f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:36.949094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8f9cb-aa1e-49b0-b8f6-36841de1a52f None admin] RBAC: Adding query filter params (role.id=192149ed0eab4f039954e4617e7ee779, scope.project.id=cc9a9fe967674703a9776dca128bc387, user.id=464b5888738c43d1883e15171a05710c) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:36.949406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8f9cb-aa1e-49b0-b8f6-36841de1a52f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:36.951457 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8f9cb-aa1e-49b0-b8f6-36841de1a52f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:36.958308 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 158/318] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:52:36 2022] GET /identity/v3/role_assignments?user.id=464b5888738c43d1883e15171a05710c&scope.project.id=cc9a9fe967674703a9776dca128bc387&role.id=192149ed0eab4f039954e4617e7ee779 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:37.194931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1951f4-4fba-40ba-923b-24b93c86fd97 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:37.195317 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1951f4-4fba-40ba-923b-24b93c86fd97 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:37.195498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1951f4-4fba-40ba-923b-24b93c86fd97 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:37.197030 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 161/319] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:37.203310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920016c-a022-4e9f-9ae8-d64275cff6db None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:37.203797 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920016c-a022-4e9f-9ae8-d64275cff6db None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:37.204083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920016c-a022-4e9f-9ae8-d64275cff6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:37.223122 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-1920016c-a022-4e9f-9ae8-d64275cff6db None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:37.224348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1920016c-a022-4e9f-9ae8-d64275cff6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:37.252525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:37.305999 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 159/320] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:37.329981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1951f4-4fba-40ba-923b-24b93c86fd97 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:37.363036 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:37.438645 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1951f4-4fba-40ba-923b-24b93c86fd97 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:37.473885 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e87a0-73c9-44d5-8a3b-4477fa72971f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:37.474609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e87a0-73c9-44d5-8a3b-4477fa72971f None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:37.474896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e87a0-73c9-44d5-8a3b-4477fa72971f None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:37.475169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e87a0-73c9-44d5-8a3b-4477fa72971f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:37.476326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e87a0-73c9-44d5-8a3b-4477fa72971f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:37.477881 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e87a0-73c9-44d5-8a3b-4477fa72971f None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:37.479870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 162/321] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:52:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:37.486442 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920016c-a022-4e9f-9ae8-d64275cff6db None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:37.588594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920016c-a022-4e9f-9ae8-d64275cff6db None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:37.626123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f819d29-2605-4f5f-91ce-3edf87b7201d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:37.626834 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f819d29-2605-4f5f-91ce-3edf87b7201d None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:37.627121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f819d29-2605-4f5f-91ce-3edf87b7201d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:37.627396 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f819d29-2605-4f5f-91ce-3edf87b7201d None admin] PATH_INFO: `/v3/groups` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:37.628101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f819d29-2605-4f5f-91ce-3edf87b7201d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:37.629557 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f819d29-2605-4f5f-91ce-3edf87b7201d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:37.696525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0857e-0a77-42df-824c-25a01257f639 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:37.696816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0857e-0a77-42df-824c-25a01257f639 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:37.697124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0857e-0a77-42df-824c-25a01257f639 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:37.698569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 163/322] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:52:37.704225 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bd7a9-a628-4910-b631-608359c05acd None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:37.705512 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bd7a9-a628-4910-b631-608359c05acd None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:37.705512 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bd7a9-a628-4910-b631-608359c05acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:37.723862 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-0b2bd7a9-a628-4910-b631-608359c05acd None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:37.725137 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0b2bd7a9-a628-4910-b631-608359c05acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:37.732450 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 160/323] 10.222.0.83 () {60 vars in 1134 bytes} [Mon Jul 18 14:52:37 2022] POST /identity/v3/groups => generated 226 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:52:37.824194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 164/324] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:52:37.847685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f819d29-2605-4f5f-91ce-3edf87b7201d None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:37.873631 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:37.964503 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f819d29-2605-4f5f-91ce-3edf87b7201d None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:38.004225 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e5430-9aa5-43d1-8314-5a6dcdfb74b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:38.005031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e5430-9aa5-43d1-8314-5a6dcdfb74b4 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:38.005317 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e5430-9aa5-43d1-8314-5a6dcdfb74b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:38.005602 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e5430-9aa5-43d1-8314-5a6dcdfb74b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:38.018276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e5430-9aa5-43d1-8314-5a6dcdfb74b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:38.026217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-618e5430-9aa5-43d1-8314-5a6dcdfb74b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:38.026217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:38.026217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:38.026217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:38.026217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:38.026217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:38.026217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:38.026217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:38.026217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:38.026217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:38.026217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:38.026217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:38.026217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:38.026217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:38.026217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:38.026217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:38.027161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:38.027161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:38.027161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:38.027161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:38.027161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:38.027161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:38.027161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:38.027161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:38.027161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:38.027161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:38.027161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:38.027161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:38.027161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:38.027161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:38.027161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:38.027161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:38.028083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:38.028083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:38.028083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:38.028083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:38.028083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:38.028083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:38.028083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:38.028083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:38.028083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:38.028083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:38.028083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:38.028083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:38.028083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 161/325] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:38.034244 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bd7a9-a628-4910-b631-608359c05acd None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:38.067727 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:38.148690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bd7a9-a628-4910-b631-608359c05acd None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:38.190730 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1473e82-739e-4128-b8cb-98aa3b848111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:38.191478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1473e82-739e-4128-b8cb-98aa3b848111 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:38.191773 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1473e82-739e-4128-b8cb-98aa3b848111 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:38.192052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1473e82-739e-4128-b8cb-98aa3b848111 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:38.192845 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1473e82-739e-4128-b8cb-98aa3b848111 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:38.193154 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1473e82-739e-4128-b8cb-98aa3b848111 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:38.194655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1473e82-739e-4128-b8cb-98aa3b848111 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:38.205829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 165/326] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:38.211869 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e5430-9aa5-43d1-8314-5a6dcdfb74b4 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:38.311101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:38.332214 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e5430-9aa5-43d1-8314-5a6dcdfb74b4 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:38.372538 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae33ab-7f49-4c71-b341-115ebf4ad02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:38.373534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae33ab-7f49-4c71-b341-115ebf4ad02d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:38.373819 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae33ab-7f49-4c71-b341-115ebf4ad02d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:38.374093 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae33ab-7f49-4c71-b341-115ebf4ad02d None admin] PATH_INFO: `/v3/users/03ef64d794a14609b9516d48fd4a962c` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:38.375528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae33ab-7f49-4c71-b341-115ebf4ad02d None admin] RBAC: Authorizing `identity:get_user(user_id=03ef64d794a14609b9516d48fd4a962c)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:38.377041 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae33ab-7f49-4c71-b341-115ebf4ad02d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:38.378785 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 162/327] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:38 2022] GET /identity/v3/users/03ef64d794a14609b9516d48fd4a962c => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:38.384677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1473e82-739e-4128-b8cb-98aa3b848111 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:38.499559 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:38.512997 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1473e82-739e-4128-b8cb-98aa3b848111 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:38.561090 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e37a83-d838-44c8-8c5b-de45ed403b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:38.562426 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e37a83-d838-44c8-8c5b-de45ed403b39 None admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:38.562954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e37a83-d838-44c8-8c5b-de45ed403b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:38.563459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e37a83-d838-44c8-8c5b-de45ed403b39 None admin] PATH_INFO: `/v3/system/users/03ef64d794a14609b9516d48fd4a962c/roles/192149ed0eab4f039954e4617e7ee779` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:38.565244 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e37a83-d838-44c8-8c5b-de45ed403b39 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=03ef64d794a14609b9516d48fd4a962c, role_id=192149ed0eab4f039954e4617e7ee779)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:38.566892 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e37a83-d838-44c8-8c5b-de45ed403b39 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:38.630349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 166/328] 10.222.0.83 () {58 vars in 1334 bytes} [Mon Jul 18 14:52:38 2022] PUT /identity/v3/system/users/03ef64d794a14609b9516d48fd4a962c/roles/192149ed0eab4f039954e4617e7ee779 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:52:38.817944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e342276-0b06-4dc3-bd6b-e98a45ed9ff2 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:38.818473 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e342276-0b06-4dc3-bd6b-e98a45ed9ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:38.818473 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e342276-0b06-4dc3-bd6b-e98a45ed9ff2 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:38.820002 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 163/329] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:38.826723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda660f-cea3-4482-a375-40895e8181b0 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:38.827651 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda660f-cea3-4482-a375-40895e8181b0 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:38.828162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda660f-cea3-4482-a375-40895e8181b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:38.847607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-8cda660f-cea3-4482-a375-40895e8181b0 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:38.849481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-8cda660f-cea3-4482-a375-40895e8181b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:38.938701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 167/330] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:38.960693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e342276-0b06-4dc3-bd6b-e98a45ed9ff2 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:39.112671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e342276-0b06-4dc3-bd6b-e98a45ed9ff2 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:39.169679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053fc32-1ed5-47d2-8f01-bc00948e92a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:39.170508 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053fc32-1ed5-47d2-8f01-bc00948e92a5 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:39.170820 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053fc32-1ed5-47d2-8f01-bc00948e92a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:39.171096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053fc32-1ed5-47d2-8f01-bc00948e92a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:39.187516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053fc32-1ed5-47d2-8f01-bc00948e92a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:39.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-3053fc32-1ed5-47d2-8f01-bc00948e92a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:39.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:39.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:39.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:39.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:39.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:39.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:39.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:39.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:39.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:39.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:39.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:39.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:39.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:39.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:39.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:39.196462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:39.196462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:39.196462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:39.196462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:39.196462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:39.196462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:39.196462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:39.196462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:39.196462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:39.196462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:39.196462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:39.196462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:39.196462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:39.196462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:39.196462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:39.196462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:39.197278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:39.197278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:39.197278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:39.197278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:39.197278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:39.197278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:39.197278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:39.197278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:39.197278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:39.197278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:39.197278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:39.197278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:39.197278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 164/331] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:39.205615 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda660f-cea3-4482-a375-40895e8181b0 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:39.328499 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:39.347152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda660f-cea3-4482-a375-40895e8181b0 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:39.392257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00b243-1d7a-47f1-8866-01faf82d3487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:39.392854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00b243-1d7a-47f1-8866-01faf82d3487 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:39.393216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00b243-1d7a-47f1-8866-01faf82d3487 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:39.393496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00b243-1d7a-47f1-8866-01faf82d3487 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:39.394269 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea00b243-1d7a-47f1-8866-01faf82d3487 None admin] RBAC: Adding query filter params (name=member) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:39.394594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea00b243-1d7a-47f1-8866-01faf82d3487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:39.396027 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea00b243-1d7a-47f1-8866-01faf82d3487 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:39.411480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 168/332] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:39.417353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053fc32-1ed5-47d2-8f01-bc00948e92a5 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:39.441457 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:39.583465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053fc32-1ed5-47d2-8f01-bc00948e92a5 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:39.627567 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f43db1-1b38-4888-bde4-8c2ee36a6fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:39.628495 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f43db1-1b38-4888-bde4-8c2ee36a6fec None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:39.628778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f43db1-1b38-4888-bde4-8c2ee36a6fec None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:39.629051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f43db1-1b38-4888-bde4-8c2ee36a6fec None admin] PATH_INFO: `/v3/users/750f4772f93b4df080f4cd9bbf03a3fb` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:39.635636 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f43db1-1b38-4888-bde4-8c2ee36a6fec None admin] RBAC: Authorizing `identity:get_user(user_id=750f4772f93b4df080f4cd9bbf03a3fb)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:39.637451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f43db1-1b38-4888-bde4-8c2ee36a6fec None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:39.639613 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 165/333] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:39 2022] GET /identity/v3/users/750f4772f93b4df080f4cd9bbf03a3fb => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:39.646153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00b243-1d7a-47f1-8866-01faf82d3487 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:39.804030 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00b243-1d7a-47f1-8866-01faf82d3487 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:39.850308 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821545b0-30f9-40b2-ada5-e9f27940d7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:39.851904 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821545b0-30f9-40b2-ada5-e9f27940d7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:39.852415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821545b0-30f9-40b2-ada5-e9f27940d7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:39.852934 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821545b0-30f9-40b2-ada5-e9f27940d7a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:39.854015 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821545b0-30f9-40b2-ada5-e9f27940d7a9 None admin] RBAC: Adding query filter params (role.id=3ab0ee91b3284a65a0dd5696a6250883, user.id=750f4772f93b4df080f4cd9bbf03a3fb, scope.system=all) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:39.854572 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821545b0-30f9-40b2-ada5-e9f27940d7a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:39.856829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821545b0-30f9-40b2-ada5-e9f27940d7a9 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:39.878873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 169/334] 10.222.0.83 () {56 vars in 1308 bytes} [Mon Jul 18 14:52:39 2022] GET /identity/v3/role_assignments?user.id=750f4772f93b4df080f4cd9bbf03a3fb&scope.system=all&role.id=3ab0ee91b3284a65a0dd5696a6250883 => generated 510 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:40.906016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848d175-56b6-4fb3-a82e-5f9f17d754f5 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:40.906668 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848d175-56b6-4fb3-a82e-5f9f17d754f5 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:40.906668 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848d175-56b6-4fb3-a82e-5f9f17d754f5 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:40.908574 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 166/335] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:40.915627 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea01bc-11b3-438b-bcf1-3772e2e3ba76 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:40.916132 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea01bc-11b3-438b-bcf1-3772e2e3ba76 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:40.916290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea01bc-11b3-438b-bcf1-3772e2e3ba76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:40.927854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:40.944143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-1bea01bc-11b3-438b-bcf1-3772e2e3ba76 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:40.945362 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-1bea01bc-11b3-438b-bcf1-3772e2e3ba76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:41.044538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 170/336] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:41.085549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848d175-56b6-4fb3-a82e-5f9f17d754f5 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:41.191219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:41.220345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848d175-56b6-4fb3-a82e-5f9f17d754f5 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:41.269200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b04410-bfdf-4ec7-bc9c-54ced88052f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:41.269200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b04410-bfdf-4ec7-bc9c-54ced88052f2 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:41.269200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b04410-bfdf-4ec7-bc9c-54ced88052f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:41.269200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b04410-bfdf-4ec7-bc9c-54ced88052f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:41.281476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b04410-bfdf-4ec7-bc9c-54ced88052f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:41.289732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-d7b04410-bfdf-4ec7-bc9c-54ced88052f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:41.289732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:41.289732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:41.289732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:41.289732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:41.289732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:41.289732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:41.289732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:41.289732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:41.289732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:41.289732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:41.289732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:41.289732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:41.289732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:41.289732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:41.289732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:41.290750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:41.290750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:41.290750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:41.290750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:41.290750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:41.290750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:41.290750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:41.290750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:41.290750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:41.290750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:41.290750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:41.290750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:41.290750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:41.290750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:41.290750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:41.290750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:41.291579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:41.291579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:41.291579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:41.291579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:41.291579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:41.291579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:41.291579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:41.291579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:41.291579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:41.291579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:41.291579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:41.291579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:41.291579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 167/337] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:41.298091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea01bc-11b3-438b-bcf1-3772e2e3ba76 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:41.436347 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea01bc-11b3-438b-bcf1-3772e2e3ba76 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:41.483753 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bff1d-bab6-4fa5-8d4e-15d059153f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:41.484698 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bff1d-bab6-4fa5-8d4e-15d059153f00 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:41.484981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bff1d-bab6-4fa5-8d4e-15d059153f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:41.485334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bff1d-bab6-4fa5-8d4e-15d059153f00 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:41.486106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bff1d-bab6-4fa5-8d4e-15d059153f00 None admin] RBAC: Adding query filter params (name=member) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:41.486413 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bff1d-bab6-4fa5-8d4e-15d059153f00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:41.487853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bff1d-bab6-4fa5-8d4e-15d059153f00 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:41.489453 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:41.503035 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 171/338] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:41.508406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b04410-bfdf-4ec7-bc9c-54ced88052f2 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:41.594991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2525cb-54fa-4e9b-b66c-cd07cc5f9a70 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:41.595303 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2525cb-54fa-4e9b-b66c-cd07cc5f9a70 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:41.595544 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2525cb-54fa-4e9b-b66c-cd07cc5f9a70 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:41.597269 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 172/339] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:41.603006 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de66ae-353e-46cd-a98a-a8623e42a55c None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:41.603295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de66ae-353e-46cd-a98a-a8623e42a55c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:41.603576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de66ae-353e-46cd-a98a-a8623e42a55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:41.624704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-92de66ae-353e-46cd-a98a-a8623e42a55c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:41.626680 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-92de66ae-353e-46cd-a98a-a8623e42a55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:41.682781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b04410-bfdf-4ec7-bc9c-54ced88052f2 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:41.742304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 173/340] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:52:41.745517 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a35932-7263-4fb4-b9ce-1fbcfbd3c844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:41.747107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a35932-7263-4fb4-b9ce-1fbcfbd3c844 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:41.747810 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a35932-7263-4fb4-b9ce-1fbcfbd3c844 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:41.748373 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a35932-7263-4fb4-b9ce-1fbcfbd3c844 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:41.758238 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:41.790108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de66ae-353e-46cd-a98a-a8623e42a55c None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:41.849801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a35932-7263-4fb4-b9ce-1fbcfbd3c844 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:41.851749 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a35932-7263-4fb4-b9ce-1fbcfbd3c844 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:42.013776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 168/341] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:42.019481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a35932-7263-4fb4-b9ce-1fbcfbd3c844 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:42.019657 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de66ae-353e-46cd-a98a-a8623e42a55c None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:42.048423 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:42.056730 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b19c3-d5b9-4ec4-a714-d9c62671b1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:42.057785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b19c3-d5b9-4ec4-a714-d9c62671b1ed None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:42.058248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b19c3-d5b9-4ec4-a714-d9c62671b1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:42.058746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b19c3-d5b9-4ec4-a714-d9c62671b1ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:42.070956 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:42.072123 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924b19c3-d5b9-4ec4-a714-d9c62671b1ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:42.081584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-924b19c3-d5b9-4ec4-a714-d9c62671b1ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:42.081584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:42.081584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:42.081584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:42.081584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:42.081584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:42.081584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:42.081584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:42.081584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:42.081584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:42.081584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:42.081584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:42.081584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:42.081584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:42.081584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:42.081584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:42.082738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:42.082738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:42.082738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:42.082738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:42.082738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:42.082738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:42.082738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:42.082738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:42.082738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:42.082738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:42.082738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:42.082738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:42.082738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:42.082738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:42.082738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:42.082738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:42.083787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:42.083787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:42.083787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:42.083787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:42.083787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:42.083787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:42.083787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:42.083787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:42.083787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:42.083787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:42.083787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:42.083787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:42.089902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 174/342] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:52:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:42.093145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b19c3-d5b9-4ec4-a714-d9c62671b1ed None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:42.177808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a35932-7263-4fb4-b9ce-1fbcfbd3c844 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:42.227749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b19c3-d5b9-4ec4-a714-d9c62671b1ed None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:42.239505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde4c0b-7364-479b-af5b-723b2f35bb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:42.240385 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde4c0b-7364-479b-af5b-723b2f35bb83 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:42.240670 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde4c0b-7364-479b-af5b-723b2f35bb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:42.240949 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde4c0b-7364-479b-af5b-723b2f35bb83 None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:42.242197 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde4c0b-7364-479b-af5b-723b2f35bb83 None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:42.243696 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde4c0b-7364-479b-af5b-723b2f35bb83 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:42.245797 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 169/343] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:42 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:42.255891 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde4c0b-7364-479b-af5b-723b2f35bb83 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:42.288273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4fe67-3f4c-4f24-a408-c8914e965585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:42.288969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4fe67-3f4c-4f24-a408-c8914e965585 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:42.289314 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4fe67-3f4c-4f24-a408-c8914e965585 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:42.289599 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4fe67-3f4c-4f24-a408-c8914e965585 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:42.290900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4fe67-3f4c-4f24-a408-c8914e965585 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:42.291450 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4fe67-3f4c-4f24-a408-c8914e965585 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:42.295133 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4fe67-3f4c-4f24-a408-c8914e965585 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:42.311139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 175/344] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:52:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:42.312850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402a72f-8beb-4a3b-b694-e4caadb9ac5c None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:42.313378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402a72f-8beb-4a3b-b694-e4caadb9ac5c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:42.313891 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402a72f-8beb-4a3b-b694-e4caadb9ac5c None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:42.315504 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 176/345] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:42.317030 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64310be-eda3-49f8-bba0-31db920950cb None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:42.317546 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64310be-eda3-49f8-bba0-31db920950cb None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:42.318163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64310be-eda3-49f8-bba0-31db920950cb None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:42.319828 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 177/346] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:42.320886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64310be-eda3-49f8-bba0-31db920950cb None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:42.400739 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde4c0b-7364-479b-af5b-723b2f35bb83 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:42.403598 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:42.414395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:42.457770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260eb20-9477-405c-b2f1-0d9ffa076b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:42.458658 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260eb20-9477-405c-b2f1-0d9ffa076b1c None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:42.458949 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260eb20-9477-405c-b2f1-0d9ffa076b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:42.459233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260eb20-9477-405c-b2f1-0d9ffa076b1c None admin] PATH_INFO: `/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:42.472780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260eb20-9477-405c-b2f1-0d9ffa076b1c None admin] RBAC: Authorizing `identity:get_group(group_id=58aaffd186e74fb8a3cf1f4124a157e1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:42.474291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260eb20-9477-405c-b2f1-0d9ffa076b1c None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:42.476223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 170/347] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Jul 18 14:52:42 2022] GET /identity/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:42.477775 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02ca99-f62b-48ed-8da7-04467b5a47b9 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:42.478004 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02ca99-f62b-48ed-8da7-04467b5a47b9 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:42.478306 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02ca99-f62b-48ed-8da7-04467b5a47b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:42.488885 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64310be-eda3-49f8-bba0-31db920950cb None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:42.509194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-ba02ca99-f62b-48ed-8da7-04467b5a47b9 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:42.510679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ba02ca99-f62b-48ed-8da7-04467b5a47b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:42.532211 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655e506-f1e5-4370-9185-164a8eb41f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:42.533261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655e506-f1e5-4370-9185-164a8eb41f6d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:42.533570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655e506-f1e5-4370-9185-164a8eb41f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:42.533887 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655e506-f1e5-4370-9185-164a8eb41f6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:42.615448 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 171/348] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:42.617051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01d517-1928-4bd8-8817-427146796b95 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:42.617531 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01d517-1928-4bd8-8817-427146796b95 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:42.618020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01d517-1928-4bd8-8817-427146796b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:42.637814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-ab01d517-1928-4bd8-8817-427146796b95 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:42.639388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ab01d517-1928-4bd8-8817-427146796b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:42.696722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655e506-f1e5-4370-9185-164a8eb41f6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:42.698213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655e506-f1e5-4370-9185-164a8eb41f6d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:42.740880 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 172/349] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:42.742432 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01d517-1928-4bd8-8817-427146796b95 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:42.777440 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:42.797499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:42.838148 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 178/350] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:42.843508 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655e506-f1e5-4370-9185-164a8eb41f6d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:42.898050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01d517-1928-4bd8-8817-427146796b95 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:42.947605 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1d40a-4005-49cb-bdc1-7fa413944463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:42.948454 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1d40a-4005-49cb-bdc1-7fa413944463 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:42.948807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1d40a-4005-49cb-bdc1-7fa413944463 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:42.949090 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1d40a-4005-49cb-bdc1-7fa413944463 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:42.951372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1d40a-4005-49cb-bdc1-7fa413944463 None admin] RBAC: Adding query filter params (group.id=58aaffd186e74fb8a3cf1f4124a157e1, role.id=3ab0ee91b3284a65a0dd5696a6250883, scope.project.id=10cc8d215a4045f1b83e4120a718db23) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:42.951932 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1d40a-4005-49cb-bdc1-7fa413944463 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:42.953495 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1d40a-4005-49cb-bdc1-7fa413944463 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:42.959874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 173/351] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Jul 18 14:52:42 2022] GET /identity/v3/role_assignments?group.id=58aaffd186e74fb8a3cf1f4124a157e1&scope.project.id=10cc8d215a4045f1b83e4120a718db23&role.id=3ab0ee91b3284a65a0dd5696a6250883 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:42.960715 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1d40a-4005-49cb-bdc1-7fa413944463 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:42.990359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655e506-f1e5-4370-9185-164a8eb41f6d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:43.045131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78af03f-1e9c-404b-82b4-444d52cde8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:43.045998 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78af03f-1e9c-404b-82b4-444d52cde8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:43.046290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78af03f-1e9c-404b-82b4-444d52cde8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:43.046593 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78af03f-1e9c-404b-82b4-444d52cde8d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:43.068488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78af03f-1e9c-404b-82b4-444d52cde8d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:43.078421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-f78af03f-1e9c-404b-82b4-444d52cde8d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:43.078421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:43.078421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:43.078421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:43.078421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:43.078421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:43.078421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:43.078421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:43.078421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:43.078421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:43.078421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:43.078421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:43.078421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:43.078421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:43.078421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:43.078421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:43.079635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:43.079635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:43.079635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:43.079635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:43.079635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:43.079635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:43.079635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:43.079635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:43.079635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:43.079635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:43.079635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:43.079635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:43.079635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:43.079635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:43.079635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:43.079635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:43.080479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:43.080479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:43.080479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:43.080479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:43.080479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:43.080479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:43.080479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:43.080479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:43.080479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:43.080479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:43.080479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:52:43.080479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:43.083503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 179/352] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:43.085229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206ff58-4d00-4d92-850b-7124f897bd7d None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:43.085860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206ff58-4d00-4d92-850b-7124f897bd7d None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:43.086358 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206ff58-4d00-4d92-850b-7124f897bd7d None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:43.088340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 180/353] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:43.089362 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206ff58-4d00-4d92-850b-7124f897bd7d None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:43.105442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:43.125893 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:43.133985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1d40a-4005-49cb-bdc1-7fa413944463 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:43.183562 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b533abe-eb70-4d77-b70e-4b08aff1276b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:43.185082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b533abe-eb70-4d77-b70e-4b08aff1276b None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:43.185373 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b533abe-eb70-4d77-b70e-4b08aff1276b None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:43.185659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b533abe-eb70-4d77-b70e-4b08aff1276b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:43.204598 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b533abe-eb70-4d77-b70e-4b08aff1276b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:43.215253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-9b533abe-eb70-4d77-b70e-4b08aff1276b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:43.215253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:43.215253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:43.215253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:43.215253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:43.215253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:43.215253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:43.215253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:43.215253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:43.215253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:43.215253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:43.215253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:43.215253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:43.215253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:43.215253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:43.215253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:43.216296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:43.216296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:43.216296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:43.216296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:43.216296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:43.216296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:43.216296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:43.216296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:43.216296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:43.216296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:43.216296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:43.216296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:43.216296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:43.216296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:43.216296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:43.216296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:43.217106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:43.217106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:43.217106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:43.217106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:43.217106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:43.217106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:43.217106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:43.217106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:43.217106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:43.217106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:43.217106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:43.217106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:43.217106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 174/354] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:43.218656 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e61523-f1fc-4721-9521-9f8fc81b70e5 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:43.218940 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e61523-f1fc-4721-9521-9f8fc81b70e5 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:43.219226 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e61523-f1fc-4721-9521-9f8fc81b70e5 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:43.221065 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 175/355] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:43.221768 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e61523-f1fc-4721-9521-9f8fc81b70e5 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:43.255370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206ff58-4d00-4d92-850b-7124f897bd7d None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:43.311879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10496901-1531-4c1b-af96-6724185b8116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:43.313714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10496901-1531-4c1b-af96-6724185b8116 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:43.314574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10496901-1531-4c1b-af96-6724185b8116 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:43.315086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10496901-1531-4c1b-af96-6724185b8116 None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:43.316806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10496901-1531-4c1b-af96-6724185b8116 None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:43.318964 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10496901-1531-4c1b-af96-6724185b8116 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:43.321124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 181/356] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:43 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:43.322618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50613af-c90d-4d3a-bf9d-88f152e77fda None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:43.323129 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50613af-c90d-4d3a-bf9d-88f152e77fda None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:43.323619 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50613af-c90d-4d3a-bf9d-88f152e77fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:43.343968 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-c50613af-c90d-4d3a-bf9d-88f152e77fda None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:43.346058 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c50613af-c90d-4d3a-bf9d-88f152e77fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:43.374350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e61523-f1fc-4721-9521-9f8fc81b70e5 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:43.416857 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601b542-ebe7-4a71-b410-84ed9be586fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:43.417539 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601b542-ebe7-4a71-b410-84ed9be586fa None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:43.417826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601b542-ebe7-4a71-b410-84ed9be586fa None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:43.418331 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601b542-ebe7-4a71-b410-84ed9be586fa None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:43.419071 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601b542-ebe7-4a71-b410-84ed9be586fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:43.419381 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601b542-ebe7-4a71-b410-84ed9be586fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:43.420921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601b542-ebe7-4a71-b410-84ed9be586fa None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:43.432475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 176/357] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:43.432475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601b542-ebe7-4a71-b410-84ed9be586fa None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:43.446034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 182/358] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:43.448418 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f65973-cfd2-4ad2-8e2f-0b620e64afe8 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:43.448715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f65973-cfd2-4ad2-8e2f-0b620e64afe8 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:43.449001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f65973-cfd2-4ad2-8e2f-0b620e64afe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:43.455800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:43.461168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:43.470662 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-f4f65973-cfd2-4ad2-8e2f-0b620e64afe8 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:43.472590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-f4f65973-cfd2-4ad2-8e2f-0b620e64afe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:43.575552 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 183/359] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:43.576783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f65973-cfd2-4ad2-8e2f-0b620e64afe8 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:43.581296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601b542-ebe7-4a71-b410-84ed9be586fa None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:43.626653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5bf8e-b4fa-4516-8eef-e069af707a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:43.627338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5bf8e-b4fa-4516-8eef-e069af707a93 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:43.627650 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5bf8e-b4fa-4516-8eef-e069af707a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:43.627927 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5bf8e-b4fa-4516-8eef-e069af707a93 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:43.628688 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5bf8e-b4fa-4516-8eef-e069af707a93 None admin] RBAC: Adding query filter params (name=member) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:43.628996 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5bf8e-b4fa-4516-8eef-e069af707a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:43.630413 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5bf8e-b4fa-4516-8eef-e069af707a93 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:43.639955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 177/360] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:43.641224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5bf8e-b4fa-4516-8eef-e069af707a93 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:43.724827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f65973-cfd2-4ad2-8e2f-0b620e64afe8 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:43.751535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:43.775151 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0c1be-a378-4183-87ac-47f0e95deaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:43.776040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0c1be-a378-4183-87ac-47f0e95deaf4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:43.777232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0c1be-a378-4183-87ac-47f0e95deaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:43.778198 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0c1be-a378-4183-87ac-47f0e95deaf4 None admin] PATH_INFO: `/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:43.780214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa0c1be-a378-4183-87ac-47f0e95deaf4 None admin] RBAC: Authorizing `identity:get_group(group_id=58aaffd186e74fb8a3cf1f4124a157e1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:43.781283 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa0c1be-a378-4183-87ac-47f0e95deaf4 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:43.784361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 184/361] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Jul 18 14:52:43 2022] GET /identity/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:43.784361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0c1be-a378-4183-87ac-47f0e95deaf4 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:43.789704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:43.796942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5bf8e-b4fa-4516-8eef-e069af707a93 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:43.850362 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd14e55-e542-45c4-b54a-5f212e1d16d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:43.851587 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd14e55-e542-45c4-b54a-5f212e1d16d6 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:43.852051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd14e55-e542-45c4-b54a-5f212e1d16d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:43.852471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd14e55-e542-45c4-b54a-5f212e1d16d6 None admin] PATH_INFO: `/v3/users/03ef64d794a14609b9516d48fd4a962c` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:43.853889 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd14e55-e542-45c4-b54a-5f212e1d16d6 None admin] RBAC: Authorizing `identity:get_user(user_id=03ef64d794a14609b9516d48fd4a962c)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:43.855415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd14e55-e542-45c4-b54a-5f212e1d16d6 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:43.857220 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 178/362] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:52:43 2022] GET /identity/v3/users/03ef64d794a14609b9516d48fd4a962c => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:43.858013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd14e55-e542-45c4-b54a-5f212e1d16d6 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:43.950805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0c1be-a378-4183-87ac-47f0e95deaf4 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:44.014637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0102a52-27b0-456c-a6cd-8dca49f288e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:44.015653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd14e55-e542-45c4-b54a-5f212e1d16d6 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:44.015768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0102a52-27b0-456c-a6cd-8dca49f288e9 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:44.016176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0102a52-27b0-456c-a6cd-8dca49f288e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:44.016576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0102a52-27b0-456c-a6cd-8dca49f288e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:44.036173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0102a52-27b0-456c-a6cd-8dca49f288e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:44.048949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-a0102a52-27b0-456c-a6cd-8dca49f288e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:44.048949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:44.048949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:44.048949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:44.048949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:44.048949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:44.048949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:44.048949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:44.048949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:44.048949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:44.048949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:44.048949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:44.048949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:44.048949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:44.048949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:44.048949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:44.050384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:44.050384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:44.050384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:44.050384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:44.050384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:44.050384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:44.050384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:44.050384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:44.050384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:44.050384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:44.050384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:44.050384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:44.050384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:44.050384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:44.050384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:44.050384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:44.051597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:44.051597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:44.051597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:44.051597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:44.051597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:44.051597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:44.051597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:44.051597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:44.051597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:44.051597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:44.051597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:44.051597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:44.051597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 185/363] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:52:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:44.052530 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0102a52-27b0-456c-a6cd-8dca49f288e9 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:44.069298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b790952-c56e-44d3-a120-3fa30ad6084d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:44.070501 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b790952-c56e-44d3-a120-3fa30ad6084d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:44.071005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b790952-c56e-44d3-a120-3fa30ad6084d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:44.071269 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b790952-c56e-44d3-a120-3fa30ad6084d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:44.086359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b790952-c56e-44d3-a120-3fa30ad6084d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:44.095444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-4b790952-c56e-44d3-a120-3fa30ad6084d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:44.095444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:44.095444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:44.095444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:44.095444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:44.095444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:44.095444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:44.095444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:44.095444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:44.095444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:44.095444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:44.095444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:44.095444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:44.095444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:44.095444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:44.095444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:44.096740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:44.096740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:44.096740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:44.096740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:44.096740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:44.096740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:44.096740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:44.096740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:44.096740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:44.096740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:44.096740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:44.096740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:44.096740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:44.096740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:44.096740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:44.096740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:44.097637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:44.097637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:44.097637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:44.097637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:44.097637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:44.097637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:44.097637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:44.097637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:44.097637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:44.097637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:44.097637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:44.097637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:44.097637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 179/364] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:52:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:44.097637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b790952-c56e-44d3-a120-3fa30ad6084d None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:44.102338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:44.147091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:44.204926 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0102a52-27b0-456c-a6cd-8dca49f288e9 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:44.238852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b790952-c56e-44d3-a120-3fa30ad6084d None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:44.256702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0657ae1-3ea6-45b8-8495-f84cb557090c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:44.257373 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0657ae1-3ea6-45b8-8495-f84cb557090c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:44.257839 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0657ae1-3ea6-45b8-8495-f84cb557090c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:44.258125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0657ae1-3ea6-45b8-8495-f84cb557090c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:44.299278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0736a6-45aa-4c25-b9a5-c915c1d03efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:44.299864 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0736a6-45aa-4c25-b9a5-c915c1d03efd None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:44.300149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0736a6-45aa-4c25-b9a5-c915c1d03efd None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:44.301395 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0736a6-45aa-4c25-b9a5-c915c1d03efd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:44.301395 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0736a6-45aa-4c25-b9a5-c915c1d03efd None admin] RBAC: Adding query filter params (group.id=58aaffd186e74fb8a3cf1f4124a157e1, role.id=a82e24db1e3c4f868a586e122dae393e, scope.project.id=10cc8d215a4045f1b83e4120a718db23) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:44.301593 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0736a6-45aa-4c25-b9a5-c915c1d03efd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:44.303904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0736a6-45aa-4c25-b9a5-c915c1d03efd None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:44.311566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 180/365] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Jul 18 14:52:44 2022] GET /identity/v3/role_assignments?group.id=58aaffd186e74fb8a3cf1f4124a157e1&scope.project.id=10cc8d215a4045f1b83e4120a718db23&role.id=a82e24db1e3c4f868a586e122dae393e => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:44.312877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0736a6-45aa-4c25-b9a5-c915c1d03efd None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:44.367979 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0657ae1-3ea6-45b8-8495-f84cb557090c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:44.369528 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0657ae1-3ea6-45b8-8495-f84cb557090c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:44.413392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:44.471610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:44.482054 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0736a6-45aa-4c25-b9a5-c915c1d03efd None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:44.522509 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 186/366] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:44.523748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0657ae1-3ea6-45b8-8495-f84cb557090c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:44.530914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f0a07-43c7-49f1-a996-4009766c79b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:44.531586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f0a07-43c7-49f1-a996-4009766c79b4 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:44.532053 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f0a07-43c7-49f1-a996-4009766c79b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:44.532346 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f0a07-43c7-49f1-a996-4009766c79b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:44.533194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f0a07-43c7-49f1-a996-4009766c79b4 None admin] RBAC: Adding query filter params (role.id=192149ed0eab4f039954e4617e7ee779, user.id=03ef64d794a14609b9516d48fd4a962c, scope.system=all) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:44.533514 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f0a07-43c7-49f1-a996-4009766c79b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:44.536099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f0a07-43c7-49f1-a996-4009766c79b4 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:44.553379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 181/367] 10.222.0.83 () {56 vars in 1308 bytes} [Mon Jul 18 14:52:44 2022] GET /identity/v3/role_assignments?user.id=03ef64d794a14609b9516d48fd4a962c&scope.system=all&role.id=192149ed0eab4f039954e4617e7ee779 => generated 510 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:44.555505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f0a07-43c7-49f1-a996-4009766c79b4 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:44.680059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0657ae1-3ea6-45b8-8495-f84cb557090c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:44.698742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f0a07-43c7-49f1-a996-4009766c79b4 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:44.739045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f49a76-e008-449c-bebb-41bad5041c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:44.739750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f49a76-e008-449c-bebb-41bad5041c42 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:44.740037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f49a76-e008-449c-bebb-41bad5041c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:44.740312 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f49a76-e008-449c-bebb-41bad5041c42 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:44.742709 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:44.743310 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f49a76-e008-449c-bebb-41bad5041c42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:44.743667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f49a76-e008-449c-bebb-41bad5041c42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:44.745127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f49a76-e008-449c-bebb-41bad5041c42 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:44.764301 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5c507-d9ba-4c1a-87cb-1f6543a1eb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:44.765055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5c507-d9ba-4c1a-87cb-1f6543a1eb02 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:44.765344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5c507-d9ba-4c1a-87cb-1f6543a1eb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:44.765624 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5c507-d9ba-4c1a-87cb-1f6543a1eb02 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:44.766406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5c507-d9ba-4c1a-87cb-1f6543a1eb02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:44.766752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5c507-d9ba-4c1a-87cb-1f6543a1eb02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:44.768981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 187/368] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:44.770588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5c507-d9ba-4c1a-87cb-1f6543a1eb02 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:44.772890 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f49a76-e008-449c-bebb-41bad5041c42 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:44.791475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 182/369] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:52:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:44.792408 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5c507-d9ba-4c1a-87cb-1f6543a1eb02 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:44.842054 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:44.954959 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f49a76-e008-449c-bebb-41bad5041c42 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:44.983404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5c507-d9ba-4c1a-87cb-1f6543a1eb02 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:45.001713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafbaeb-183e-42c8-ad9f-fe0cb90f148e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:45.002633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafbaeb-183e-42c8-ad9f-fe0cb90f148e None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:45.002995 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafbaeb-183e-42c8-ad9f-fe0cb90f148e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:45.003287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafbaeb-183e-42c8-ad9f-fe0cb90f148e None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:45.004567 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafbaeb-183e-42c8-ad9f-fe0cb90f148e None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:45.006082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafbaeb-183e-42c8-ad9f-fe0cb90f148e None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:45.008117 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 188/370] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:44 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:45.008706 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafbaeb-183e-42c8-ad9f-fe0cb90f148e None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:45.030372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aefe7-72e2-4ea1-baec-45824155d4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:45.031502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aefe7-72e2-4ea1-baec-45824155d4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:45.032064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aefe7-72e2-4ea1-baec-45824155d4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:45.032566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aefe7-72e2-4ea1-baec-45824155d4e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:45.100948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:45.117200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33aefe7-72e2-4ea1-baec-45824155d4e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:45.119226 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33aefe7-72e2-4ea1-baec-45824155d4e3 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:45.153089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:45.161949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafbaeb-183e-42c8-ad9f-fe0cb90f148e None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:45.206022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed163b-2fdf-4f20-b99d-64b52bc55502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:45.206863 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed163b-2fdf-4f20-b99d-64b52bc55502 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:45.207154 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed163b-2fdf-4f20-b99d-64b52bc55502 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:45.207507 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed163b-2fdf-4f20-b99d-64b52bc55502 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:45.246242 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 183/371] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:45.248141 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aefe7-72e2-4ea1-baec-45824155d4e3 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:45.290059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed163b-2fdf-4f20-b99d-64b52bc55502 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:45.291992 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed163b-2fdf-4f20-b99d-64b52bc55502 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:45.362500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:45.377530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aefe7-72e2-4ea1-baec-45824155d4e3 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:45.402979 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:45.411626 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 189/372] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:45.412745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed163b-2fdf-4f20-b99d-64b52bc55502 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:45.425588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a5b20-de5c-4721-af23-1e75947fbcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:45.426667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a5b20-de5c-4721-af23-1e75947fbcdb None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:45.426997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a5b20-de5c-4721-af23-1e75947fbcdb None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:45.427332 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a5b20-de5c-4721-af23-1e75947fbcdb None admin] PATH_INFO: `/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:45.428766 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a5b20-de5c-4721-af23-1e75947fbcdb None admin] RBAC: Authorizing `identity:get_group(group_id=58aaffd186e74fb8a3cf1f4124a157e1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:45.430419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a5b20-de5c-4721-af23-1e75947fbcdb None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:45.432545 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 184/373] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Jul 18 14:52:45 2022] GET /identity/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:45.433997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a5b20-de5c-4721-af23-1e75947fbcdb None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:45.551238 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed163b-2fdf-4f20-b99d-64b52bc55502 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:45.573954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a5b20-de5c-4721-af23-1e75947fbcdb None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:45.602995 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643e833-78c0-4f11-ac29-f858ba60555e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:45.604012 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643e833-78c0-4f11-ac29-f858ba60555e None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:45.604301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643e833-78c0-4f11-ac29-f858ba60555e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:45.604700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643e833-78c0-4f11-ac29-f858ba60555e None admin] PATH_INFO: `/v3/projects/8015bf50f271424ca62c239a2df319f8` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:45.606647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643e833-78c0-4f11-ac29-f858ba60555e None admin] RBAC: Authorizing `identity:get_project(project_id=8015bf50f271424ca62c239a2df319f8)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:45.608128 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643e833-78c0-4f11-ac29-f858ba60555e None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:45.610736 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 190/374] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:45 2022] GET /identity/v3/projects/8015bf50f271424ca62c239a2df319f8 => generated 344 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:45.612081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d75db-52ab-43d2-a7cc-38c4aeb4db5e None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:45.612564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d75db-52ab-43d2-a7cc-38c4aeb4db5e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:45.612797 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d75db-52ab-43d2-a7cc-38c4aeb4db5e None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:45.614323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 191/375] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:45.615068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d75db-52ab-43d2-a7cc-38c4aeb4db5e None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:45.627871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f834d-e78b-4d9d-8ce3-82a9e22d87a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:45.628781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f834d-e78b-4d9d-8ce3-82a9e22d87a0 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:45.629066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f834d-e78b-4d9d-8ce3-82a9e22d87a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:45.629444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f834d-e78b-4d9d-8ce3-82a9e22d87a0 None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:45.631306 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f834d-e78b-4d9d-8ce3-82a9e22d87a0 None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:45.632798 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f834d-e78b-4d9d-8ce3-82a9e22d87a0 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:45.634929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 185/376] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:45 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:45.635927 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f834d-e78b-4d9d-8ce3-82a9e22d87a0 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:45.669092 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:45.720055 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:45.766465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f834d-e78b-4d9d-8ce3-82a9e22d87a0 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:45.782907 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d75db-52ab-43d2-a7cc-38c4aeb4db5e None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:45.821253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ff740-b30d-485b-99b8-35cbeb009e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:45.822803 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ff740-b30d-485b-99b8-35cbeb009e6a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:45.822803 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ff740-b30d-485b-99b8-35cbeb009e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:45.822803 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ff740-b30d-485b-99b8-35cbeb009e6a None admin] PATH_INFO: `/v3/groups/f6e248a998054c808dd111543fc10c1f` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:45.829563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ff740-b30d-485b-99b8-35cbeb009e6a None admin] RBAC: Authorizing `identity:get_group(group_id=f6e248a998054c808dd111543fc10c1f)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:45.831143 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ff740-b30d-485b-99b8-35cbeb009e6a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:45.832910 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 186/377] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Jul 18 14:52:45 2022] GET /identity/v3/groups/f6e248a998054c808dd111543fc10c1f => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:45.834172 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4d07b-d3be-4fa9-9bf6-7dd2513145a5 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:45.835344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4d07b-d3be-4fa9-9bf6-7dd2513145a5 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:45.835344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4d07b-d3be-4fa9-9bf6-7dd2513145a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:45.848372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2ece2-b3b9-49e0-8a55-8c149793375e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:45.849057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2ece2-b3b9-49e0-8a55-8c149793375e None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:45.849347 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2ece2-b3b9-49e0-8a55-8c149793375e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:45.849670 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2ece2-b3b9-49e0-8a55-8c149793375e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:45.850613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2ece2-b3b9-49e0-8a55-8c149793375e None admin] RBAC: Adding query filter params (group.id=58aaffd186e74fb8a3cf1f4124a157e1, role.id=3ab0ee91b3284a65a0dd5696a6250883, scope.project.id=cc9a9fe967674703a9776dca128bc387) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:45.852367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2ece2-b3b9-49e0-8a55-8c149793375e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:45.853340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2ece2-b3b9-49e0-8a55-8c149793375e None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:45.854404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-80c4d07b-d3be-4fa9-9bf6-7dd2513145a5 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:45.856244 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-80c4d07b-d3be-4fa9-9bf6-7dd2513145a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:45.861759 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 192/378] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Jul 18 14:52:45 2022] GET /identity/v3/role_assignments?group.id=58aaffd186e74fb8a3cf1f4124a157e1&scope.project.id=cc9a9fe967674703a9776dca128bc387&role.id=3ab0ee91b3284a65a0dd5696a6250883 => generated 260 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:45.862475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2ece2-b3b9-49e0-8a55-8c149793375e None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:45.974761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 187/379] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:45.976234 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4d07b-d3be-4fa9-9bf6-7dd2513145a5 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:46.025442 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:46.065090 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2ece2-b3b9-49e0-8a55-8c149793375e None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:46.108625 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:46.120404 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47297aac-0611-4ed0-94e5-024aef3198c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:46.121874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47297aac-0611-4ed0-94e5-024aef3198c0 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:46.122167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47297aac-0611-4ed0-94e5-024aef3198c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:46.122451 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47297aac-0611-4ed0-94e5-024aef3198c0 None admin] PATH_INFO: `/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:46.124155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47297aac-0611-4ed0-94e5-024aef3198c0 None admin] RBAC: Authorizing `identity:get_group(group_id=58aaffd186e74fb8a3cf1f4124a157e1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:46.125651 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47297aac-0611-4ed0-94e5-024aef3198c0 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:46.127440 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 193/380] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Jul 18 14:52:45 2022] GET /identity/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:46.128352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47297aac-0611-4ed0-94e5-024aef3198c0 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:46.161042 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4d07b-d3be-4fa9-9bf6-7dd2513145a5 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:46.217956 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2289346-37ca-4060-b987-61a83411d954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:46.219072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2289346-37ca-4060-b987-61a83411d954 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:46.219591 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2289346-37ca-4060-b987-61a83411d954 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:46.220085 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2289346-37ca-4060-b987-61a83411d954 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:46.221126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2289346-37ca-4060-b987-61a83411d954 None admin] RBAC: Adding query filter params (group.id=f6e248a998054c808dd111543fc10c1f, role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=8015bf50f271424ca62c239a2df319f8) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:46.221649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2289346-37ca-4060-b987-61a83411d954 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:46.223360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2289346-37ca-4060-b987-61a83411d954 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:46.230198 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 188/381] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Jul 18 14:52:45 2022] GET /identity/v3/role_assignments?group.id=f6e248a998054c808dd111543fc10c1f&scope.project.id=8015bf50f271424ca62c239a2df319f8&role.id=5891920310b54787bd1152d736d2a3f1 => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:46.231914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2289346-37ca-4060-b987-61a83411d954 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:46.274955 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47297aac-0611-4ed0-94e5-024aef3198c0 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:46.316909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d2c62-896d-437d-a02b-9856e82d4c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:46.317327 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d2c62-896d-437d-a02b-9856e82d4c14 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:46.317619 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d2c62-896d-437d-a02b-9856e82d4c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:46.317900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d2c62-896d-437d-a02b-9856e82d4c14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:46.330859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d2c62-896d-437d-a02b-9856e82d4c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:46.331848 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:46.339683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-949d2c62-896d-437d-a02b-9856e82d4c14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:46.339683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:46.339683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:46.339683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:46.339683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:46.339683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:46.339683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:46.339683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:46.339683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:46.339683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:46.339683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:46.339683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:46.339683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:46.339683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:46.339683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:46.339683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:46.341366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:46.341366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:46.341366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:46.341366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:46.341366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:46.341366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:46.341366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:46.341366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:46.341366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:46.341366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:46.341366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:46.341366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:46.341366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:46.341366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:46.341366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:46.341366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:46.342176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:46.342176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:46.342176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:46.342176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:46.342176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:46.342176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:46.342176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:46.342176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:46.342176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:46.342176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:46.342176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:46.342176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:46.342176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 194/382] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:46.347735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d2c62-896d-437d-a02b-9856e82d4c14 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:46.357992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2289346-37ca-4060-b987-61a83411d954 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:46.397746 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b5c10-5df2-431a-8c51-919ae33b231a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:46.398918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b5c10-5df2-431a-8c51-919ae33b231a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:46.399435 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b5c10-5df2-431a-8c51-919ae33b231a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:46.399925 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b5c10-5df2-431a-8c51-919ae33b231a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:46.401093 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b5c10-5df2-431a-8c51-919ae33b231a None admin] RBAC: Adding query filter params (group.id=58aaffd186e74fb8a3cf1f4124a157e1, role.id=a82e24db1e3c4f868a586e122dae393e, scope.project.id=cc9a9fe967674703a9776dca128bc387) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:46.401651 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b5c10-5df2-431a-8c51-919ae33b231a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:46.404243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b5c10-5df2-431a-8c51-919ae33b231a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:46.411130 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 189/383] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Jul 18 14:52:46 2022] GET /identity/v3/role_assignments?group.id=58aaffd186e74fb8a3cf1f4124a157e1&scope.project.id=cc9a9fe967674703a9776dca128bc387&role.id=a82e24db1e3c4f868a586e122dae393e => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:46.413302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:46.478231 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d2c62-896d-437d-a02b-9856e82d4c14 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:46.520072 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5973cde-08db-4e7f-8095-958a7579f3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:46.520816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5973cde-08db-4e7f-8095-958a7579f3ca None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:46.521110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5973cde-08db-4e7f-8095-958a7579f3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:46.521448 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5973cde-08db-4e7f-8095-958a7579f3ca None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:46.522245 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5973cde-08db-4e7f-8095-958a7579f3ca None admin] RBAC: Adding query filter params (name=member) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:46.522589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5973cde-08db-4e7f-8095-958a7579f3ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:46.524062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5973cde-08db-4e7f-8095-958a7579f3ca None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:46.535913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 195/384] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:46.541144 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b5c10-5df2-431a-8c51-919ae33b231a None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:46.689012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b5c10-5df2-431a-8c51-919ae33b231a None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:46.749189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce34ab-0aff-4e03-a598-8eb2b64c6505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:46.750078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce34ab-0aff-4e03-a598-8eb2b64c6505 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:46.750360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce34ab-0aff-4e03-a598-8eb2b64c6505 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:46.750672 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce34ab-0aff-4e03-a598-8eb2b64c6505 None admin] PATH_INFO: `/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:46.751884 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce34ab-0aff-4e03-a598-8eb2b64c6505 None admin] RBAC: Authorizing `identity:get_group(group_id=58aaffd186e74fb8a3cf1f4124a157e1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:46.753365 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce34ab-0aff-4e03-a598-8eb2b64c6505 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:46.755413 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 190/385] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Jul 18 14:52:46 2022] GET /identity/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:46.760829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5973cde-08db-4e7f-8095-958a7579f3ca None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:46.916672 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5973cde-08db-4e7f-8095-958a7579f3ca None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:46.951108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:46.970125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a5bc4-0034-4b2e-98bd-3e672ada25ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:46.973078 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a5bc4-0034-4b2e-98bd-3e672ada25ff None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:46.973078 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a5bc4-0034-4b2e-98bd-3e672ada25ff None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:46.973078 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a5bc4-0034-4b2e-98bd-3e672ada25ff None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:46.975418 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a5bc4-0034-4b2e-98bd-3e672ada25ff None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:46.976928 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a5bc4-0034-4b2e-98bd-3e672ada25ff None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:46.978797 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 196/386] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:46 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:46.988041 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce34ab-0aff-4e03-a598-8eb2b64c6505 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:47.015985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:47.069295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056bab0-7296-474f-a462-8bb14a0d47b1 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:47.069673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056bab0-7296-474f-a462-8bb14a0d47b1 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:47.069869 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056bab0-7296-474f-a462-8bb14a0d47b1 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:47.071357 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 197/387] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:47.080131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60505b5-6703-4229-bd61-bf7716dfdbdd None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:47.080506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60505b5-6703-4229-bd61-bf7716dfdbdd None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:47.080696 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60505b5-6703-4229-bd61-bf7716dfdbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:47.128261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-a60505b5-6703-4229-bd61-bf7716dfdbdd None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:47.129572 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a60505b5-6703-4229-bd61-bf7716dfdbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:47.148808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce34ab-0aff-4e03-a598-8eb2b64c6505 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:47.203327 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f1fb8-cf24-4029-a69d-53c28bb3792f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:47.204235 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f1fb8-cf24-4029-a69d-53c28bb3792f None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:47.204524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f1fb8-cf24-4029-a69d-53c28bb3792f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:47.204803 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f1fb8-cf24-4029-a69d-53c28bb3792f None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23/groups/58aaffd186e74fb8a3cf1f4124a157e1/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:47.207691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f1fb8-cf24-4029-a69d-53c28bb3792f None admin] RBAC: Authorizing `identity:create_grant(project_id=10cc8d215a4045f1b83e4120a718db23, group_id=58aaffd186e74fb8a3cf1f4124a157e1, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:47.209193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f1fb8-cf24-4029-a69d-53c28bb3792f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:47.250361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 198/388] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:47.272775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60505b5-6703-4229-bd61-bf7716dfdbdd None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:47.340345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 191/389] 10.222.0.83 () {58 vars in 1442 bytes} [Mon Jul 18 14:52:46 2022] PUT /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23/groups/58aaffd186e74fb8a3cf1f4124a157e1/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:52:47.409316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60505b5-6703-4229-bd61-bf7716dfdbdd None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:47.434848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:47.469228 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91bdc3-e147-41b0-91e8-e22b0a38b162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:47.470062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91bdc3-e147-41b0-91e8-e22b0a38b162 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:47.470347 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91bdc3-e147-41b0-91e8-e22b0a38b162 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:47.470649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91bdc3-e147-41b0-91e8-e22b0a38b162 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:47.483690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91bdc3-e147-41b0-91e8-e22b0a38b162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:47.492602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-0b91bdc3-e147-41b0-91e8-e22b0a38b162 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:47.492602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:47.492602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:47.492602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:47.492602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:47.492602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:47.492602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:47.492602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:47.492602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:47.492602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:47.492602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:47.492602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:47.492602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:47.492602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:47.492602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:47.492602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:47.493646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:47.493646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:47.493646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:47.493646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:47.493646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:47.493646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:47.493646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:47.493646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:47.493646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:47.493646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:47.493646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:47.493646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:47.493646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:47.493646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:47.493646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:47.493646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:47.494694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:47.494694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:47.494694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:47.494694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:47.494694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:47.494694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:47.494694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:47.494694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:47.494694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:47.494694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:47.494694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:47.494694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:47.494694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 199/390] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:52:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:52:47.500810 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f1fb8-cf24-4029-a69d-53c28bb3792f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:47.617597 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:47.656525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f1fb8-cf24-4029-a69d-53c28bb3792f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:47.712412 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fcbb7-cccd-40c2-bec2-4cc8bbca2563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:47.713099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fcbb7-cccd-40c2-bec2-4cc8bbca2563 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:47.713386 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fcbb7-cccd-40c2-bec2-4cc8bbca2563 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:47.713694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fcbb7-cccd-40c2-bec2-4cc8bbca2563 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:47.714447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fcbb7-cccd-40c2-bec2-4cc8bbca2563 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:47.714932 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fcbb7-cccd-40c2-bec2-4cc8bbca2563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:47.716799 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fcbb7-cccd-40c2-bec2-4cc8bbca2563 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:47.730311 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 192/391] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:52:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:47.736518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91bdc3-e147-41b0-91e8-e22b0a38b162 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:47.879620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91bdc3-e147-41b0-91e8-e22b0a38b162 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:47.919679 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f513b33-0a4f-4dd7-9dce-903b50b5144b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:47.920600 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f513b33-0a4f-4dd7-9dce-903b50b5144b None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:47.920949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f513b33-0a4f-4dd7-9dce-903b50b5144b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:47.921237 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f513b33-0a4f-4dd7-9dce-903b50b5144b None admin] PATH_INFO: `/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:47.922562 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f513b33-0a4f-4dd7-9dce-903b50b5144b None admin] RBAC: Authorizing `identity:get_group(group_id=58aaffd186e74fb8a3cf1f4124a157e1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:47.924034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f513b33-0a4f-4dd7-9dce-903b50b5144b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:47.931718 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fcbb7-cccd-40c2-bec2-4cc8bbca2563 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:47.932788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 200/392] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Jul 18 14:52:47 2022] GET /identity/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:48.076463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fcbb7-cccd-40c2-bec2-4cc8bbca2563 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:48.127315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb689b3-2dd8-4551-8d7b-c1b6856ddba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:48.128220 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb689b3-2dd8-4551-8d7b-c1b6856ddba9 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:48.128503 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb689b3-2dd8-4551-8d7b-c1b6856ddba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:48.128779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb689b3-2dd8-4551-8d7b-c1b6856ddba9 None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:48.130114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb689b3-2dd8-4551-8d7b-c1b6856ddba9 None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:48.131603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb689b3-2dd8-4551-8d7b-c1b6856ddba9 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:48.133581 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 193/393] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:47 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:48.140052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f513b33-0a4f-4dd7-9dce-903b50b5144b None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:48.215032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:48.268023 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f513b33-0a4f-4dd7-9dce-903b50b5144b None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:48.304954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98caea4f-128e-4692-ae7d-a2b8e1f2eb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:48.305849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caea4f-128e-4692-ae7d-a2b8e1f2eb9b None admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:48.306145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caea4f-128e-4692-ae7d-a2b8e1f2eb9b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:48.306431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caea4f-128e-4692-ae7d-a2b8e1f2eb9b None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23/groups/58aaffd186e74fb8a3cf1f4124a157e1/roles/a82e24db1e3c4f868a586e122dae393e` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:48.309696 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98caea4f-128e-4692-ae7d-a2b8e1f2eb9b None admin] RBAC: Authorizing `identity:create_grant(project_id=10cc8d215a4045f1b83e4120a718db23, group_id=58aaffd186e74fb8a3cf1f4124a157e1, role_id=a82e24db1e3c4f868a586e122dae393e)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:48.311252 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98caea4f-128e-4692-ae7d-a2b8e1f2eb9b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:48.394383 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 201/394] 10.222.0.83 () {58 vars in 1442 bytes} [Mon Jul 18 14:52:48 2022] PUT /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23/groups/58aaffd186e74fb8a3cf1f4124a157e1/roles/a82e24db1e3c4f868a586e122dae393e => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:52:48.537203 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb845bd-694d-4686-a56f-0c33f6801be7 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:48.538311 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb845bd-694d-4686-a56f-0c33f6801be7 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:48.538850 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb845bd-694d-4686-a56f-0c33f6801be7 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:48.541008 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 194/395] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:48.547519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f1dfe-fa1f-4d80-84dd-341c26a300c4 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:48.548050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f1dfe-fa1f-4d80-84dd-341c26a300c4 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:48.548164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f1dfe-fa1f-4d80-84dd-341c26a300c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:48.571534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b42f1dfe-fa1f-4d80-84dd-341c26a300c4 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:48.572809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b42f1dfe-fa1f-4d80-84dd-341c26a300c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:48.682901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 202/396] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:48.705512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb845bd-694d-4686-a56f-0c33f6801be7 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:48.708833 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658386a8-8e67-4dae-b050-20ac6763dd58 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:48.709393 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658386a8-8e67-4dae-b050-20ac6763dd58 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:48.709888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658386a8-8e67-4dae-b050-20ac6763dd58 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:48.710068 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:48.711953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 203/397] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:48.717955 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec6d20-70d9-4a38-af76-d50152428068 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:48.718268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec6d20-70d9-4a38-af76-d50152428068 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:48.718527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec6d20-70d9-4a38-af76-d50152428068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:48.740214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-a2ec6d20-70d9-4a38-af76-d50152428068 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:48.741471 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a2ec6d20-70d9-4a38-af76-d50152428068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:48.785054 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:48.861602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 204/398] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:52:48.876276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb845bd-694d-4686-a56f-0c33f6801be7 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:48.904018 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec6d20-70d9-4a38-af76-d50152428068 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:48.946702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558dbb84-c227-4647-940c-c3295a06ffba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:48.947884 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dbb84-c227-4647-940c-c3295a06ffba None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:48.949459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dbb84-c227-4647-940c-c3295a06ffba None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:48.950040 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dbb84-c227-4647-940c-c3295a06ffba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:48.975622 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558dbb84-c227-4647-940c-c3295a06ffba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:48.991601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-558dbb84-c227-4647-940c-c3295a06ffba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:48.991601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:48.991601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:48.991601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:48.991601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:48.991601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:48.991601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:48.991601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:48.991601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:48.991601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:48.991601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:48.991601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:48.991601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:48.991601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:48.991601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:48.991601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:48.992787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:48.992787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:48.992787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:48.992787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:48.992787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:48.992787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:48.992787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:48.992787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:48.992787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:48.992787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:48.992787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:48.992787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:48.992787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:48.992787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:48.992787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:48.992787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:48.993620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:48.993620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:48.993620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:48.993620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:48.993620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:48.993620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:48.993620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:48.993620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:48.993620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:48.993620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:48.993620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:48.993620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:48.998950 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 195/399] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:49.000752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7de03-6bab-408c-a41b-267260b157f1 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:49.001076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7de03-6bab-408c-a41b-267260b157f1 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:49.001359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7de03-6bab-408c-a41b-267260b157f1 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:49.004354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 196/400] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:49.005821 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be7de03-6bab-408c-a41b-267260b157f1 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:49.073070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec6d20-70d9-4a38-af76-d50152428068 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:49.078159 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:49.116902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122eb90a-4d89-4204-b4fb-f96358e06bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:49.118092 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122eb90a-4d89-4204-b4fb-f96358e06bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:49.118500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122eb90a-4d89-4204-b4fb-f96358e06bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:49.118900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122eb90a-4d89-4204-b4fb-f96358e06bb0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:49.124267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be7de03-6bab-408c-a41b-267260b157f1 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:49.134361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122eb90a-4d89-4204-b4fb-f96358e06bb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:49.146105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-122eb90a-4d89-4204-b4fb-f96358e06bb0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:49.146105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:49.146105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:49.146105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:49.146105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:49.146105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:49.146105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:49.146105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:49.146105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:49.146105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:49.146105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:49.146105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:49.146105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:49.146105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:49.146105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:49.146105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:49.147169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:49.147169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:49.147169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:49.147169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:49.147169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:49.147169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:49.147169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:49.147169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:49.147169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:49.147169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:49.147169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:49.147169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:49.147169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:49.147169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:49.147169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:49.147169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:49.147991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:49.147991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:49.147991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:49.147991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:49.147991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:49.147991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:49.147991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:49.147991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:49.147991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:49.147991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:49.147991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:49.147991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:49.147991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 205/401] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:52:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:52:49.149576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce8185-5b73-4c1d-ab3d-bff92116ca93 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:49.149963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce8185-5b73-4c1d-ab3d-bff92116ca93 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:49.150324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce8185-5b73-4c1d-ab3d-bff92116ca93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:49.169982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd94a7d-bcd6-4068-8a55-8eee348b6816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:49.171081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd94a7d-bcd6-4068-8a55-8eee348b6816 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:49.171629 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd94a7d-bcd6-4068-8a55-8eee348b6816 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:49.172114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd94a7d-bcd6-4068-8a55-8eee348b6816 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:49.173101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd94a7d-bcd6-4068-8a55-8eee348b6816 None admin] RBAC: Adding query filter params (name=member) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:49.173503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-22ce8185-5b73-4c1d-ab3d-bff92116ca93 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:49.173631 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd94a7d-bcd6-4068-8a55-8eee348b6816 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:49.175106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-22ce8185-5b73-4c1d-ab3d-bff92116ca93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:49.175402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd94a7d-bcd6-4068-8a55-8eee348b6816 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:49.191002 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 197/402] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:49.192116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd94a7d-bcd6-4068-8a55-8eee348b6816 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:49.198957 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:49.282868 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 206/403] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:49.288506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce8185-5b73-4c1d-ab3d-bff92116ca93 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:49.339796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd94a7d-bcd6-4068-8a55-8eee348b6816 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:49.379372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:49.392778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c86056-5ab5-44b2-83fb-5ecf5e843eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:49.394056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c86056-5ab5-44b2-83fb-5ecf5e843eac None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:49.394502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c86056-5ab5-44b2-83fb-5ecf5e843eac None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:49.394961 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c86056-5ab5-44b2-83fb-5ecf5e843eac None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:49.395885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c86056-5ab5-44b2-83fb-5ecf5e843eac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:49.396389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c86056-5ab5-44b2-83fb-5ecf5e843eac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:49.398372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c86056-5ab5-44b2-83fb-5ecf5e843eac None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:49.411648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 198/404] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:49.413480 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c86056-5ab5-44b2-83fb-5ecf5e843eac None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:49.482322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce8185-5b73-4c1d-ab3d-bff92116ca93 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:49.524820 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea164d-e19c-49bf-b17b-05eb6c7acb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:49.525982 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea164d-e19c-49bf-b17b-05eb6c7acb4d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:49.526378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea164d-e19c-49bf-b17b-05eb6c7acb4d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:49.526796 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea164d-e19c-49bf-b17b-05eb6c7acb4d None admin] PATH_INFO: `/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:49.528558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea164d-e19c-49bf-b17b-05eb6c7acb4d None admin] RBAC: Authorizing `identity:get_group(group_id=58aaffd186e74fb8a3cf1f4124a157e1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:49.530735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea164d-e19c-49bf-b17b-05eb6c7acb4d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:49.537858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 207/405] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Jul 18 14:52:49 2022] GET /identity/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1 => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:49.538874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea164d-e19c-49bf-b17b-05eb6c7acb4d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:49.554767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c86056-5ab5-44b2-83fb-5ecf5e843eac None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:49.566424 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:49.598995 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7d9a1-9a96-43c0-aece-3b556947fdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:49.601090 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7d9a1-9a96-43c0-aece-3b556947fdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:49.601722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7d9a1-9a96-43c0-aece-3b556947fdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:49.602340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7d9a1-9a96-43c0-aece-3b556947fdf7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:49.617132 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7d9a1-9a96-43c0-aece-3b556947fdf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:49.626871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-78d7d9a1-9a96-43c0-aece-3b556947fdf7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:49.626871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:49.626871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:49.626871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:49.626871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:49.626871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:49.626871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:49.626871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:49.626871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:49.626871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:49.626871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:49.626871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:49.626871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:49.626871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:49.626871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:49.626871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:49.627796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:49.627796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:49.627796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:49.627796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:49.627796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:49.627796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:49.627796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:49.627796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:49.627796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:49.627796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:49.627796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:49.627796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:49.627796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:49.627796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:49.627796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:49.627796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:49.628623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:49.628623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:49.628623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:49.628623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:49.628623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:49.628623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:49.628623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:49.628623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:49.628623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:49.628623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:49.628623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:49.628623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:49.631506 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 199/406] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:52:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:49.632663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7d9a1-9a96-43c0-aece-3b556947fdf7 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:49.673167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea164d-e19c-49bf-b17b-05eb6c7acb4d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:49.689828 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:49.721442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18ca03-885f-45c6-9afb-7b66ef30370b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:49.722422 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18ca03-885f-45c6-9afb-7b66ef30370b None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:49.722760 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18ca03-885f-45c6-9afb-7b66ef30370b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:49.723042 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18ca03-885f-45c6-9afb-7b66ef30370b None admin] PATH_INFO: `/v3/groups/f6e248a998054c808dd111543fc10c1f` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:49.724979 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18ca03-885f-45c6-9afb-7b66ef30370b None admin] RBAC: Authorizing `identity:get_group(group_id=f6e248a998054c808dd111543fc10c1f)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:49.727118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18ca03-885f-45c6-9afb-7b66ef30370b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:49.729311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 208/407] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Jul 18 14:52:49 2022] GET /identity/v3/groups/f6e248a998054c808dd111543fc10c1f => generated 229 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:49.730669 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18ca03-885f-45c6-9afb-7b66ef30370b None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:49.754073 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7d9a1-9a96-43c0-aece-3b556947fdf7 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:49.795101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9535514-cd79-4bfc-aaca-2ed9daf7397e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:49.796383 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9535514-cd79-4bfc-aaca-2ed9daf7397e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:49.796914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9535514-cd79-4bfc-aaca-2ed9daf7397e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:49.797411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9535514-cd79-4bfc-aaca-2ed9daf7397e None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:49.800546 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9535514-cd79-4bfc-aaca-2ed9daf7397e None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:49.802364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9535514-cd79-4bfc-aaca-2ed9daf7397e None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:49.804822 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 200/408] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:49 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:49.805933 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9535514-cd79-4bfc-aaca-2ed9daf7397e None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:49.850903 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:49.865568 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18ca03-885f-45c6-9afb-7b66ef30370b None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:49.909109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cdb02-abe2-4db4-82c0-f8cb8f4ae11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:49.909840 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cdb02-abe2-4db4-82c0-f8cb8f4ae11c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:49.910128 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cdb02-abe2-4db4-82c0-f8cb8f4ae11c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:49.910409 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cdb02-abe2-4db4-82c0-f8cb8f4ae11c None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:49.911196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cdb02-abe2-4db4-82c0-f8cb8f4ae11c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:49.911506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cdb02-abe2-4db4-82c0-f8cb8f4ae11c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:49.913181 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cdb02-abe2-4db4-82c0-f8cb8f4ae11c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:49.924138 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 209/409] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:52:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:49.931445 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cdb02-abe2-4db4-82c0-f8cb8f4ae11c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:49.935653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9535514-cd79-4bfc-aaca-2ed9daf7397e None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:49.954342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:49.977751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a11994-3bf1-4037-8442-c14848a7e212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:49.979222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a11994-3bf1-4037-8442-c14848a7e212 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:49.979667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a11994-3bf1-4037-8442-c14848a7e212 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:49.980280 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a11994-3bf1-4037-8442-c14848a7e212 None admin] PATH_INFO: `/v3/projects/8015bf50f271424ca62c239a2df319f8` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:49.982091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a11994-3bf1-4037-8442-c14848a7e212 None admin] RBAC: Authorizing `identity:get_project(project_id=8015bf50f271424ca62c239a2df319f8)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:49.984103 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a11994-3bf1-4037-8442-c14848a7e212 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:49.986443 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 201/410] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:49 2022] GET /identity/v3/projects/8015bf50f271424ca62c239a2df319f8 => generated 344 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:49.987699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a11994-3bf1-4037-8442-c14848a7e212 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:50.051951 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cdb02-abe2-4db4-82c0-f8cb8f4ae11c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:50.094219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e3dca-740b-4ac5-af8e-57b2910ef7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:50.095202 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e3dca-740b-4ac5-af8e-57b2910ef7f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:50.095492 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e3dca-740b-4ac5-af8e-57b2910ef7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:50.095938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e3dca-740b-4ac5-af8e-57b2910ef7f1 None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387/groups/58aaffd186e74fb8a3cf1f4124a157e1/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:50.098068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e3dca-740b-4ac5-af8e-57b2910ef7f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc9a9fe967674703a9776dca128bc387, group_id=58aaffd186e74fb8a3cf1f4124a157e1, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:50.099759 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e3dca-740b-4ac5-af8e-57b2910ef7f1 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:50.108458 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a11994-3bf1-4037-8442-c14848a7e212 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:50.154146 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146de81d-cf9d-47f7-a5b0-8942a2e3c090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:50.155051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146de81d-cf9d-47f7-a5b0-8942a2e3c090 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:50.155340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146de81d-cf9d-47f7-a5b0-8942a2e3c090 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:50.155618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146de81d-cf9d-47f7-a5b0-8942a2e3c090 None admin] PATH_INFO: `/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:50.156828 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146de81d-cf9d-47f7-a5b0-8942a2e3c090 None admin] RBAC: Authorizing `identity:get_group(group_id=58aaffd186e74fb8a3cf1f4124a157e1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:50.158376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146de81d-cf9d-47f7-a5b0-8942a2e3c090 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:50.165801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 202/411] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Jul 18 14:52:49 2022] GET /identity/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:50.165801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146de81d-cf9d-47f7-a5b0-8942a2e3c090 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:50.221053 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 210/412] 10.222.0.83 () {58 vars in 1442 bytes} [Mon Jul 18 14:52:49 2022] PUT /identity/v3/projects/cc9a9fe967674703a9776dca128bc387/groups/58aaffd186e74fb8a3cf1f4124a157e1/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:52:50.221856 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e3dca-740b-4ac5-af8e-57b2910ef7f1 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:50.241929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:50.247836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:50.324878 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146de81d-cf9d-47f7-a5b0-8942a2e3c090 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:50.368092 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e3dca-740b-4ac5-af8e-57b2910ef7f1 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:50.376139 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b4658-5715-46f3-8996-4092425a9407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:50.378199 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b4658-5715-46f3-8996-4092425a9407 None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:50.378789 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b4658-5715-46f3-8996-4092425a9407 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:50.379287 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b4658-5715-46f3-8996-4092425a9407 None admin] PATH_INFO: `/v3/projects/8015bf50f271424ca62c239a2df319f8/groups/f6e248a998054c808dd111543fc10c1f/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:50.382825 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b4658-5715-46f3-8996-4092425a9407 None admin] RBAC: Authorizing `identity:create_grant(project_id=8015bf50f271424ca62c239a2df319f8, group_id=f6e248a998054c808dd111543fc10c1f, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:50.384835 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b4658-5715-46f3-8996-4092425a9407 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:50.420311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aba309-3266-4230-a73b-06af3ab467f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:50.421379 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aba309-3266-4230-a73b-06af3ab467f7 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:50.421525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aba309-3266-4230-a73b-06af3ab467f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:50.421778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aba309-3266-4230-a73b-06af3ab467f7 None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:50.423152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aba309-3266-4230-a73b-06af3ab467f7 None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:50.424662 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aba309-3266-4230-a73b-06af3ab467f7 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:50.426485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 211/413] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:50 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:50.428043 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ee40c-74a9-4ecb-83ad-4dca6f708b6e None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:50.428359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ee40c-74a9-4ecb-83ad-4dca6f708b6e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:50.428646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ee40c-74a9-4ecb-83ad-4dca6f708b6e None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:50.429967 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 212/414] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:50.435066 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ee40c-74a9-4ecb-83ad-4dca6f708b6e None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:50.463655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 203/415] 10.222.0.83 () {58 vars in 1442 bytes} [Mon Jul 18 14:52:50 2022] PUT /identity/v3/projects/8015bf50f271424ca62c239a2df319f8/groups/f6e248a998054c808dd111543fc10c1f/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:52:50.465846 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fa0e6-c636-4fd7-92f9-ad5398b64ac7 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:50.466378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fa0e6-c636-4fd7-92f9-ad5398b64ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:50.467224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fa0e6-c636-4fd7-92f9-ad5398b64ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:50.491620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-df6fa0e6-c636-4fd7-92f9-ad5398b64ac7 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:50.494017 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-df6fa0e6-c636-4fd7-92f9-ad5398b64ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:50.549913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:50.614610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ee40c-74a9-4ecb-83ad-4dca6f708b6e None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:50.634095 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 204/416] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:50.677373 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fa0e6-c636-4fd7-92f9-ad5398b64ac7 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:50.710349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e1a6b-8c83-440c-a1aa-06467907c73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:50.711647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e1a6b-8c83-440c-a1aa-06467907c73f None admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:50.712159 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e1a6b-8c83-440c-a1aa-06467907c73f None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:50.712667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e1a6b-8c83-440c-a1aa-06467907c73f None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387/groups/58aaffd186e74fb8a3cf1f4124a157e1/roles/a82e24db1e3c4f868a586e122dae393e` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:50.719405 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e1a6b-8c83-440c-a1aa-06467907c73f None admin] RBAC: Authorizing `identity:create_grant(project_id=cc9a9fe967674703a9776dca128bc387, group_id=58aaffd186e74fb8a3cf1f4124a157e1, role_id=a82e24db1e3c4f868a586e122dae393e)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:50.721258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e1a6b-8c83-440c-a1aa-06467907c73f None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:50.737245 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:50.812253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fa0e6-c636-4fd7-92f9-ad5398b64ac7 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:50.819940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 213/417] 10.222.0.83 () {58 vars in 1442 bytes} [Mon Jul 18 14:52:50 2022] PUT /identity/v3/projects/cc9a9fe967674703a9776dca128bc387/groups/58aaffd186e74fb8a3cf1f4124a157e1/roles/a82e24db1e3c4f868a586e122dae393e => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:52:50.821578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a687d-ac76-4cbb-92fc-0c7aec46c9d5 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:50.822311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a687d-ac76-4cbb-92fc-0c7aec46c9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:50.822716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a687d-ac76-4cbb-92fc-0c7aec46c9d5 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:50.825071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 214/418] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:50.842966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c52ac-5aee-414b-a550-3464a832b7bb None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:50.843255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c52ac-5aee-414b-a550-3464a832b7bb None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:50.843715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c52ac-5aee-414b-a550-3464a832b7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:50.859655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02978241-56e4-4210-9a06-ca6f54bfe9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:50.861155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02978241-56e4-4210-9a06-ca6f54bfe9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:50.861769 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02978241-56e4-4210-9a06-ca6f54bfe9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:50.863717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02978241-56e4-4210-9a06-ca6f54bfe9b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:50.878170 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-4e6c52ac-5aee-414b-a550-3464a832b7bb None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:50.879063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02978241-56e4-4210-9a06-ca6f54bfe9b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:50.879534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-4e6c52ac-5aee-414b-a550-3464a832b7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:50.889339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-02978241-56e4-4210-9a06-ca6f54bfe9b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:50.889339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:50.889339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:50.889339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:50.889339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:50.889339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:50.889339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:50.889339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:50.889339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:50.889339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:50.889339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:50.889339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:50.889339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:50.889339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:50.889339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:50.889339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:50.890329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:50.890329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:50.890329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:50.890329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:50.890329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:50.890329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:50.890329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:50.890329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:50.890329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:50.890329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:50.890329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:50.890329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:50.890329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:50.890329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:50.890329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:50.890329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:50.891207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:50.891207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:50.891207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:50.891207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:50.891207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:50.891207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:50.891207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:50.891207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:50.891207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:50.891207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:50.891207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:50.891207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:50.894383 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 205/419] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:50.900331 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02978241-56e4-4210-9a06-ca6f54bfe9b1 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:51.048843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 215/420] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:52:51.089534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c52ac-5aee-414b-a550-3464a832b7bb None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:51.108360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02978241-56e4-4210-9a06-ca6f54bfe9b1 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:51.110508 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:51.161092 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32765d85-e652-4dfd-930e-f80d21c48107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:51.162347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32765d85-e652-4dfd-930e-f80d21c48107 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:51.163078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32765d85-e652-4dfd-930e-f80d21c48107 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:51.163599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32765d85-e652-4dfd-930e-f80d21c48107 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:51.164577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32765d85-e652-4dfd-930e-f80d21c48107 None admin] RBAC: Adding query filter params (name=member) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:51.165095 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32765d85-e652-4dfd-930e-f80d21c48107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:51.166874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32765d85-e652-4dfd-930e-f80d21c48107 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:51.181025 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 206/421] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:51.187760 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:51.190759 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32765d85-e652-4dfd-930e-f80d21c48107 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:51.242196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c52ac-5aee-414b-a550-3464a832b7bb None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:51.320681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea7860-43e7-4008-b6ae-efc7af5f8336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:51.322008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea7860-43e7-4008-b6ae-efc7af5f8336 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:51.322504 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea7860-43e7-4008-b6ae-efc7af5f8336 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:51.323125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea7860-43e7-4008-b6ae-efc7af5f8336 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:51.366475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32765d85-e652-4dfd-930e-f80d21c48107 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:51.379255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea7860-43e7-4008-b6ae-efc7af5f8336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:51.396139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-aaea7860-43e7-4008-b6ae-efc7af5f8336 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:51.396139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:51.396139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:51.396139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:51.396139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:51.396139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:51.396139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:51.396139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:51.396139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:51.396139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:51.396139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:51.396139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:51.396139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:51.396139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:51.396139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:51.396139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:51.397144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:51.397144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:51.397144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:51.397144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:51.397144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:51.397144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:51.397144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:51.397144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:51.397144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:51.397144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:51.397144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:51.397144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:51.397144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:51.397144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:51.397144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:51.397144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:51.397963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:51.397963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:51.397963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:51.397963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:51.397963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:51.397963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:51.397963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:51.397963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:51.397963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:51.397963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:51.397963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:51.397963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:51.397963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 216/422] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:52:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:51.405370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea7860-43e7-4008-b6ae-efc7af5f8336 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:51.409660 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edb326-1fa6-4806-88fa-e7bb5386e4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:51.410337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edb326-1fa6-4806-88fa-e7bb5386e4ed None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:51.410660 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edb326-1fa6-4806-88fa-e7bb5386e4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:51.410943 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edb326-1fa6-4806-88fa-e7bb5386e4ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:51.467564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:51.491929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edb326-1fa6-4806-88fa-e7bb5386e4ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:51.493458 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edb326-1fa6-4806-88fa-e7bb5386e4ed None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:51.583277 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea7860-43e7-4008-b6ae-efc7af5f8336 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:51.616926 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:51.630837 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819dd5c-197b-4ab5-a351-5a31a701b8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:51.631521 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819dd5c-197b-4ab5-a351-5a31a701b8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:51.631801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819dd5c-197b-4ab5-a351-5a31a701b8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:51.632072 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819dd5c-197b-4ab5-a351-5a31a701b8a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:51.632826 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819dd5c-197b-4ab5-a351-5a31a701b8a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:51.633133 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819dd5c-197b-4ab5-a351-5a31a701b8a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:51.634561 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819dd5c-197b-4ab5-a351-5a31a701b8a4 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:51.649528 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 217/423] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:52:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:51.651631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819dd5c-197b-4ab5-a351-5a31a701b8a4 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:51.702448 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 207/424] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 14:52:51.711911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edb326-1fa6-4806-88fa-e7bb5386e4ed None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:51.803096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819dd5c-197b-4ab5-a351-5a31a701b8a4 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:51.837297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:51.846619 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73585a3-6e10-47d2-a06a-62acbc683861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:51.848205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73585a3-6e10-47d2-a06a-62acbc683861 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:51.848958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73585a3-6e10-47d2-a06a-62acbc683861 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:51.849707 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73585a3-6e10-47d2-a06a-62acbc683861 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:51.930438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edb326-1fa6-4806-88fa-e7bb5386e4ed None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:51.993766 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:52.019929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2419140-73a1-4d0d-8b36-1f4c8b33cced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:52.022666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2419140-73a1-4d0d-8b36-1f4c8b33cced None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:52.023266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2419140-73a1-4d0d-8b36-1f4c8b33cced None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:52.024268 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2419140-73a1-4d0d-8b36-1f4c8b33cced None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:52.025134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73585a3-6e10-47d2-a06a-62acbc683861 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:52.028429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73585a3-6e10-47d2-a06a-62acbc683861 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:52.029397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2419140-73a1-4d0d-8b36-1f4c8b33cced None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:52.031133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2419140-73a1-4d0d-8b36-1f4c8b33cced None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:52.037924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 208/425] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:51 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:52.047038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2419140-73a1-4d0d-8b36-1f4c8b33cced None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:52.212148 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2419140-73a1-4d0d-8b36-1f4c8b33cced None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:52.254163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 218/426] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:52.263361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73585a3-6e10-47d2-a06a-62acbc683861 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:52.282312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477e937-e3c8-4c46-adb0-55ce709b3f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:52.283244 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477e937-e3c8-4c46-adb0-55ce709b3f9f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:52.283531 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477e937-e3c8-4c46-adb0-55ce709b3f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:52.283811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477e937-e3c8-4c46-adb0-55ce709b3f9f None admin] PATH_INFO: `/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:52.285385 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477e937-e3c8-4c46-adb0-55ce709b3f9f None admin] RBAC: Authorizing `identity:get_group(group_id=58aaffd186e74fb8a3cf1f4124a157e1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:52.286914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477e937-e3c8-4c46-adb0-55ce709b3f9f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:52.288835 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 209/427] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Jul 18 14:52:52 2022] GET /identity/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1 => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:52.289205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:52.295955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477e937-e3c8-4c46-adb0-55ce709b3f9f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:52.481273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:52.494224 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73585a3-6e10-47d2-a06a-62acbc683861 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:52.527056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477e937-e3c8-4c46-adb0-55ce709b3f9f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:52.538466 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15f594-b4d6-4f9e-935b-89991e9c1b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:52.539671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15f594-b4d6-4f9e-935b-89991e9c1b79 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:52.540109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15f594-b4d6-4f9e-935b-89991e9c1b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:52.540530 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15f594-b4d6-4f9e-935b-89991e9c1b79 None admin] PATH_INFO: `/v3/projects/10cc8d215a4045f1b83e4120a718db23` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:52.543458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15f594-b4d6-4f9e-935b-89991e9c1b79 None admin] RBAC: Authorizing `identity:get_project(project_id=10cc8d215a4045f1b83e4120a718db23)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:52.545153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15f594-b4d6-4f9e-935b-89991e9c1b79 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:52.549024 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 219/428] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:52 2022] GET /identity/v3/projects/10cc8d215a4045f1b83e4120a718db23 => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:52.555091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15f594-b4d6-4f9e-935b-89991e9c1b79 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:52.607284 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8cf9f-ecfd-4ade-8c0e-b8d85b8b0d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:52.608535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8cf9f-ecfd-4ade-8c0e-b8d85b8b0d70 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:52.609304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8cf9f-ecfd-4ade-8c0e-b8d85b8b0d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:52.609806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8cf9f-ecfd-4ade-8c0e-b8d85b8b0d70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:52.611107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8cf9f-ecfd-4ade-8c0e-b8d85b8b0d70 None admin] RBAC: Adding query filter params (group.id=58aaffd186e74fb8a3cf1f4124a157e1, role.id=3ab0ee91b3284a65a0dd5696a6250883, scope.project.id=10cc8d215a4045f1b83e4120a718db23) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:52.611637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8cf9f-ecfd-4ade-8c0e-b8d85b8b0d70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:52.613318 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8cf9f-ecfd-4ade-8c0e-b8d85b8b0d70 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:52.621638 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 210/429] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Jul 18 14:52:52 2022] GET /identity/v3/role_assignments?group.id=58aaffd186e74fb8a3cf1f4124a157e1&scope.project.id=10cc8d215a4045f1b83e4120a718db23&role.id=3ab0ee91b3284a65a0dd5696a6250883 => generated 611 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:52.801456 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15f594-b4d6-4f9e-935b-89991e9c1b79 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:52.816690 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f2915-97f2-4bf4-bb5a-dc50b857f504 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:52.817233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f2915-97f2-4bf4-bb5a-dc50b857f504 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:52.817233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f2915-97f2-4bf4-bb5a-dc50b857f504 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:52.818958 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 211/430] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:52:52.826156 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effffbce-6a4c-4c23-b3c1-915ba86c33fd None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:52.826460 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effffbce-6a4c-4c23-b3c1-915ba86c33fd None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:52.826749 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effffbce-6a4c-4c23-b3c1-915ba86c33fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:52.848071 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-effffbce-6a4c-4c23-b3c1-915ba86c33fd None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:52.849299 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-effffbce-6a4c-4c23-b3c1-915ba86c33fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:52.873815 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844768a-6a4a-4343-828b-7a7a3261d70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:52.875742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844768a-6a4a-4343-828b-7a7a3261d70d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:52.876261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844768a-6a4a-4343-828b-7a7a3261d70d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:52.876752 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844768a-6a4a-4343-828b-7a7a3261d70d None admin] PATH_INFO: `/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:52.879342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844768a-6a4a-4343-828b-7a7a3261d70d None admin] RBAC: Authorizing `identity:get_group(group_id=58aaffd186e74fb8a3cf1f4124a157e1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:52.882362 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844768a-6a4a-4343-828b-7a7a3261d70d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:52.887012 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 220/431] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Jul 18 14:52:52 2022] GET /identity/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1 => generated 226 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:52.895647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844768a-6a4a-4343-828b-7a7a3261d70d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:52.919038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:52.930038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:52.930558 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 212/432] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:52.952139 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903987cb-1e45-4226-bc4f-6aabc0ec5cc2 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:52.952475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903987cb-1e45-4226-bc4f-6aabc0ec5cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:52.952717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903987cb-1e45-4226-bc4f-6aabc0ec5cc2 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:52.958334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 213/433] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:52:52.961394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2a646-e7c3-46e4-9e9d-76f8781abb0a None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:52.961691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2a646-e7c3-46e4-9e9d-76f8781abb0a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:52.962011 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2a646-e7c3-46e4-9e9d-76f8781abb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:52.984747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-69f2a646-e7c3-46e4-9e9d-76f8781abb0a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:52.985986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-69f2a646-e7c3-46e4-9e9d-76f8781abb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:53.033849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844768a-6a4a-4343-828b-7a7a3261d70d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:53.076124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6b37b-a21e-4b1d-918e-e2ce06e6abeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:53.076707 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6b37b-a21e-4b1d-918e-e2ce06e6abeb None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:53.076999 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6b37b-a21e-4b1d-918e-e2ce06e6abeb None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:53.077291 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6b37b-a21e-4b1d-918e-e2ce06e6abeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:53.078128 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6b37b-a21e-4b1d-918e-e2ce06e6abeb None admin] RBAC: Adding query filter params (group.id=58aaffd186e74fb8a3cf1f4124a157e1, role.id=a82e24db1e3c4f868a586e122dae393e, scope.project.id=10cc8d215a4045f1b83e4120a718db23) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:53.078450 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6b37b-a21e-4b1d-918e-e2ce06e6abeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:53.080595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6b37b-a21e-4b1d-918e-e2ce06e6abeb None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:53.087798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 221/434] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Jul 18 14:52:52 2022] GET /identity/v3/role_assignments?group.id=58aaffd186e74fb8a3cf1f4124a157e1&scope.project.id=10cc8d215a4045f1b83e4120a718db23&role.id=a82e24db1e3c4f868a586e122dae393e => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:53.088796 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6b37b-a21e-4b1d-918e-e2ce06e6abeb None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:53.102936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 214/435] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:52:53.131741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2a646-e7c3-46e4-9e9d-76f8781abb0a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:53.207358 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:53.222293 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6b37b-a21e-4b1d-918e-e2ce06e6abeb None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:53.277375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2a646-e7c3-46e4-9e9d-76f8781abb0a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:53.284415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c7624-41ef-4e11-8800-58c678213e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:53.285642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c7624-41ef-4e11-8800-58c678213e99 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:53.286130 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c7624-41ef-4e11-8800-58c678213e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:53.286942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c7624-41ef-4e11-8800-58c678213e99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:53.308531 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:53.313437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c7624-41ef-4e11-8800-58c678213e99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:53.321020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f9ccf-30f4-49a8-8029-4a8d8d0d13d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:53.321832 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f9ccf-30f4-49a8-8029-4a8d8d0d13d7 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:53.322115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f9ccf-30f4-49a8-8029-4a8d8d0d13d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:53.322394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f9ccf-30f4-49a8-8029-4a8d8d0d13d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:53.325786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-c40c7624-41ef-4e11-8800-58c678213e99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:53.325786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:53.325786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:53.325786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:53.325786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:53.325786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:53.325786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:53.325786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:53.325786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:53.325786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:53.325786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:53.325786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:53.325786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:53.325786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:53.325786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:53.325786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:53.326805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:53.326805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:53.326805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:53.326805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:53.326805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:53.326805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:53.326805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:53.326805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:53.326805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:53.326805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:53.326805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:53.326805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:53.326805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:53.326805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:53.326805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:53.326805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:53.328136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:53.328136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:53.328136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:53.328136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:53.328136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:53.328136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:53.328136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:53.328136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:53.328136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:53.328136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:53.328136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:52:53.328136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:53.330726 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 222/436] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:53.337228 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f9ccf-30f4-49a8-8029-4a8d8d0d13d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:53.340929 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c7624-41ef-4e11-8800-58c678213e99 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:53.348843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-ed4f9ccf-30f4-49a8-8029-4a8d8d0d13d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:53.348843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:53.348843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:53.348843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:53.348843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:53.348843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:53.348843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:53.348843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:53.348843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:53.348843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:53.348843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:53.348843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:53.348843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:53.348843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:53.348843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:53.348843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:53.349921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:53.349921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:53.349921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:53.349921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:53.349921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:53.349921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:53.349921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:53.349921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:53.349921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:53.349921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:53.349921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:53.349921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:53.349921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:53.349921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:53.349921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:53.349921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:53.350787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:53.350787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:53.350787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:53.350787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:53.350787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:53.350787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:53.350787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:53.350787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:53.350787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:53.350787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:53.350787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:52:53.350787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:53.350787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 215/437] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:52:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:52:53.356713 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f9ccf-30f4-49a8-8029-4a8d8d0d13d7 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:53.503221 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f9ccf-30f4-49a8-8029-4a8d8d0d13d7 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:53.512068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c7624-41ef-4e11-8800-58c678213e99 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:53.549812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855b968-fbad-4ff1-a07d-26ab156bae06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:53.550492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855b968-fbad-4ff1-a07d-26ab156bae06 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:53.550824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855b968-fbad-4ff1-a07d-26ab156bae06 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:53.551189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855b968-fbad-4ff1-a07d-26ab156bae06 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:53.551865 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855b968-fbad-4ff1-a07d-26ab156bae06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:53.552227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855b968-fbad-4ff1-a07d-26ab156bae06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:53.553578 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855b968-fbad-4ff1-a07d-26ab156bae06 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:53.555752 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2951fd-67f6-4795-b009-ed35be2e6fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:53.556426 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2951fd-67f6-4795-b009-ed35be2e6fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:53.556713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2951fd-67f6-4795-b009-ed35be2e6fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:53.557287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2951fd-67f6-4795-b009-ed35be2e6fa3 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:53.557749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2951fd-67f6-4795-b009-ed35be2e6fa3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:53.558636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2951fd-67f6-4795-b009-ed35be2e6fa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:53.560831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2951fd-67f6-4795-b009-ed35be2e6fa3 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:53.565168 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 216/438] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:52:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:53.566524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b377f-ba55-4db5-886c-dba83b7ee6a7 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:53.566524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b377f-ba55-4db5-886c-dba83b7ee6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:53.566874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b377f-ba55-4db5-886c-dba83b7ee6a7 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:53.568406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 217/439] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:53.570472 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 223/440] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:52:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:53.573446 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2951fd-67f6-4795-b009-ed35be2e6fa3 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:53.574382 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267aee4-1269-485c-a8bc-2613273f0a32 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:53.574728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267aee4-1269-485c-a8bc-2613273f0a32 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:53.575015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267aee4-1269-485c-a8bc-2613273f0a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:53.595276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:53.596839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-5267aee4-1269-485c-a8bc-2613273f0a32 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:53.598085 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-5267aee4-1269-485c-a8bc-2613273f0a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:53.656793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:53.685202 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 218/441] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:52:53.687474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267aee4-1269-485c-a8bc-2613273f0a32 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:53.700897 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2951fd-67f6-4795-b009-ed35be2e6fa3 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:53.735381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf18ad0-8e98-4a0b-a325-2ec5ab1bf741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:53.736067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf18ad0-8e98-4a0b-a325-2ec5ab1bf741 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:53.736352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf18ad0-8e98-4a0b-a325-2ec5ab1bf741 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:53.736630 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf18ad0-8e98-4a0b-a325-2ec5ab1bf741 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:53.795414 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267aee4-1269-485c-a8bc-2613273f0a32 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:53.807371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf18ad0-8e98-4a0b-a325-2ec5ab1bf741 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:53.810216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf18ad0-8e98-4a0b-a325-2ec5ab1bf741 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:53.824412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:53.831767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9a2e3-06aa-43d0-bda9-f444efd31b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:53.832424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9a2e3-06aa-43d0-bda9-f444efd31b7f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:53.832709 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9a2e3-06aa-43d0-bda9-f444efd31b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:53.832991 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9a2e3-06aa-43d0-bda9-f444efd31b7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:53.880929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:53.905664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9a2e3-06aa-43d0-bda9-f444efd31b7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:53.907187 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9a2e3-06aa-43d0-bda9-f444efd31b7f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:53.915197 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 224/442] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:53.916048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf18ad0-8e98-4a0b-a325-2ec5ab1bf741 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:54.017305 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 219/443] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:54.018286 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9a2e3-06aa-43d0-bda9-f444efd31b7f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:54.024867 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf18ad0-8e98-4a0b-a325-2ec5ab1bf741 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:54.038188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:54.060138 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737512c4-b205-4330-94ac-9c37f7eaf37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:54.060952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737512c4-b205-4330-94ac-9c37f7eaf37e None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:54.061238 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737512c4-b205-4330-94ac-9c37f7eaf37e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:54.061515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737512c4-b205-4330-94ac-9c37f7eaf37e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:54.072194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737512c4-b205-4330-94ac-9c37f7eaf37e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:52:54.081831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-737512c4-b205-4330-94ac-9c37f7eaf37e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:54.081831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:54.081831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:54.081831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:54.081831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:54.081831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:54.081831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:54.081831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:54.081831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:54.081831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:54.081831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:54.081831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:54.081831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:52:54.081831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:52:54.081831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:52:54.081831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:52:54.082981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:52:54.082981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:52:54.082981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:52:54.082981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:52:54.082981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:54.082981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:54.082981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:54.082981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:54.082981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:54.082981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:54.082981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:54.082981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:54.082981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:54.082981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:54.082981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:54.082981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:54.088931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:54.088931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:54.088931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:54.088931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:54.088931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:52:54.088931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:52:54.088931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:52:54.088931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:52:54.088931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:52:54.088931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:52:54.088931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:52:54.088931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:54.088931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 225/444] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:52:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:54.088931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737512c4-b205-4330-94ac-9c37f7eaf37e None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:54.104548 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:54.129225 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9a2e3-06aa-43d0-bda9-f444efd31b7f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:54.166755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054d4e2-4ce6-439f-b793-3ce8aa2d56df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:54.167813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054d4e2-4ce6-439f-b793-3ce8aa2d56df None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:54.168108 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054d4e2-4ce6-439f-b793-3ce8aa2d56df None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:54.168392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054d4e2-4ce6-439f-b793-3ce8aa2d56df None admin] PATH_INFO: `/v3/projects/8015bf50f271424ca62c239a2df319f8` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:54.169732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054d4e2-4ce6-439f-b793-3ce8aa2d56df None admin] RBAC: Authorizing `identity:get_project(project_id=8015bf50f271424ca62c239a2df319f8)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:54.171289 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054d4e2-4ce6-439f-b793-3ce8aa2d56df None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:54.173325 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 220/445] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:54 2022] GET /identity/v3/projects/8015bf50f271424ca62c239a2df319f8 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:54.174049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054d4e2-4ce6-439f-b793-3ce8aa2d56df None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:54.196909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737512c4-b205-4330-94ac-9c37f7eaf37e None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:54.233392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d4164-1fb0-4259-9643-86c46da12aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:54.234516 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d4164-1fb0-4259-9643-86c46da12aab None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:54.234836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d4164-1fb0-4259-9643-86c46da12aab None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:54.235117 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d4164-1fb0-4259-9643-86c46da12aab None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:54.236328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d4164-1fb0-4259-9643-86c46da12aab None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:54.237793 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d4164-1fb0-4259-9643-86c46da12aab None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:54.244009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 226/446] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:54 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:54.244009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d4164-1fb0-4259-9643-86c46da12aab None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:54.283486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054d4e2-4ce6-439f-b793-3ce8aa2d56df None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:54.285938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:54.319046 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a04884-38ec-441b-b662-49e0d59ebf29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:54.319711 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a04884-38ec-441b-b662-49e0d59ebf29 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:54.319995 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a04884-38ec-441b-b662-49e0d59ebf29 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:54.320270 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a04884-38ec-441b-b662-49e0d59ebf29 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:54.321015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a04884-38ec-441b-b662-49e0d59ebf29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:54.321320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a04884-38ec-441b-b662-49e0d59ebf29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:54.322791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a04884-38ec-441b-b662-49e0d59ebf29 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:54.332553 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 221/447] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:52:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:54.333258 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a04884-38ec-441b-b662-49e0d59ebf29 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:54.346481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:54.347977 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d4164-1fb0-4259-9643-86c46da12aab None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:54.383870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c6550-fea8-4bf3-93ee-a3b259f4de71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:54.384724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c6550-fea8-4bf3-93ee-a3b259f4de71 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:54.385010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c6550-fea8-4bf3-93ee-a3b259f4de71 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:54.385310 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c6550-fea8-4bf3-93ee-a3b259f4de71 None admin] PATH_INFO: `/v3/groups/f6e248a998054c808dd111543fc10c1f` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:54.387517 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c6550-fea8-4bf3-93ee-a3b259f4de71 None admin] RBAC: Authorizing `identity:get_group(group_id=f6e248a998054c808dd111543fc10c1f)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:54.390055 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c6550-fea8-4bf3-93ee-a3b259f4de71 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:54.397740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 227/448] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Jul 18 14:52:54 2022] GET /identity/v3/groups/f6e248a998054c808dd111543fc10c1f => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:54.397740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c6550-fea8-4bf3-93ee-a3b259f4de71 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:54.444371 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a04884-38ec-441b-b662-49e0d59ebf29 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:54.480900 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03424343-3a43-4f17-9934-1d7976f5788a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:54.481877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03424343-3a43-4f17-9934-1d7976f5788a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:54.482166 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03424343-3a43-4f17-9934-1d7976f5788a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:54.482445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03424343-3a43-4f17-9934-1d7976f5788a None admin] PATH_INFO: `/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:54.483972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03424343-3a43-4f17-9934-1d7976f5788a None admin] RBAC: Authorizing `identity:get_group(group_id=58aaffd186e74fb8a3cf1f4124a157e1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:54.485462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03424343-3a43-4f17-9934-1d7976f5788a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:54.492144 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 222/449] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Jul 18 14:52:54 2022] GET /identity/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:54.493018 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03424343-3a43-4f17-9934-1d7976f5788a None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:54.501113 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c6550-fea8-4bf3-93ee-a3b259f4de71 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:54.510443 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:54.536360 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8827a6-f14a-4a72-a48b-201a6e749791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:54.537007 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8827a6-f14a-4a72-a48b-201a6e749791 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:54.537292 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8827a6-f14a-4a72-a48b-201a6e749791 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:54.537568 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8827a6-f14a-4a72-a48b-201a6e749791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:54.569939 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:54.595531 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03424343-3a43-4f17-9934-1d7976f5788a None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:54.614384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8827a6-f14a-4a72-a48b-201a6e749791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:54.615893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8827a6-f14a-4a72-a48b-201a6e749791 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:54.630653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab3754-244e-459e-83f4-31a70d836a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:54.631312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab3754-244e-459e-83f4-31a70d836a28 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:54.631594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab3754-244e-459e-83f4-31a70d836a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:54.631869 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab3754-244e-459e-83f4-31a70d836a28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:54.632701 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab3754-244e-459e-83f4-31a70d836a28 None admin] RBAC: Adding query filter params (group.id=f6e248a998054c808dd111543fc10c1f, role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=8015bf50f271424ca62c239a2df319f8) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:54.633022 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab3754-244e-459e-83f4-31a70d836a28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:54.634516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab3754-244e-459e-83f4-31a70d836a28 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:54.641235 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 223/450] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Jul 18 14:52:54 2022] GET /identity/v3/role_assignments?group.id=f6e248a998054c808dd111543fc10c1f&scope.project.id=8015bf50f271424ca62c239a2df319f8&role.id=5891920310b54787bd1152d736d2a3f1 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:54.642028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab3754-244e-459e-83f4-31a70d836a28 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:54.757848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 228/451] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:52:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:52:54.765168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8827a6-f14a-4a72-a48b-201a6e749791 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:54.770204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab3754-244e-459e-83f4-31a70d836a28 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:54.777293 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:54.818491 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb7e2c-da19-4290-b3b2-d356d1537c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:54.819048 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb7e2c-da19-4290-b3b2-d356d1537c35 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:54.819359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb7e2c-da19-4290-b3b2-d356d1537c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:54.819615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb7e2c-da19-4290-b3b2-d356d1537c35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:54.820477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb7e2c-da19-4290-b3b2-d356d1537c35 None admin] RBAC: Adding query filter params (group.id=58aaffd186e74fb8a3cf1f4124a157e1, role.id=3ab0ee91b3284a65a0dd5696a6250883, scope.project.id=cc9a9fe967674703a9776dca128bc387) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:54.820797 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb7e2c-da19-4290-b3b2-d356d1537c35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:54.822943 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb7e2c-da19-4290-b3b2-d356d1537c35 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:54.829455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 224/452] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Jul 18 14:52:54 2022] GET /identity/v3/role_assignments?group.id=58aaffd186e74fb8a3cf1f4124a157e1&scope.project.id=cc9a9fe967674703a9776dca128bc387&role.id=3ab0ee91b3284a65a0dd5696a6250883 => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:54.895848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8827a6-f14a-4a72-a48b-201a6e749791 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:54.935933 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8815be7-53fd-46ea-a55e-eba12b36bb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:54.936820 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8815be7-53fd-46ea-a55e-eba12b36bb3c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:54.937101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8815be7-53fd-46ea-a55e-eba12b36bb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:54.937377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8815be7-53fd-46ea-a55e-eba12b36bb3c None admin] PATH_INFO: `/v3/projects/cc9a9fe967674703a9776dca128bc387` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:54.938672 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8815be7-53fd-46ea-a55e-eba12b36bb3c None admin] RBAC: Authorizing `identity:get_project(project_id=cc9a9fe967674703a9776dca128bc387)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:54.940165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8815be7-53fd-46ea-a55e-eba12b36bb3c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:54.941936 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 229/453] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:52:54 2022] GET /identity/v3/projects/cc9a9fe967674703a9776dca128bc387 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:54.948314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb7e2c-da19-4290-b3b2-d356d1537c35 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:54.952080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:55.059300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb7e2c-da19-4290-b3b2-d356d1537c35 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:55.095383 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9aa9d-1393-425f-aa85-c4fd8dd216e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:55.096252 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9aa9d-1393-425f-aa85-c4fd8dd216e8 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:55.096563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9aa9d-1393-425f-aa85-c4fd8dd216e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:55.096879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9aa9d-1393-425f-aa85-c4fd8dd216e8 None admin] PATH_INFO: `/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:55.098066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9aa9d-1393-425f-aa85-c4fd8dd216e8 None admin] RBAC: Authorizing `identity:get_group(group_id=58aaffd186e74fb8a3cf1f4124a157e1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:55.099553 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9aa9d-1393-425f-aa85-c4fd8dd216e8 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:55.101211 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 225/454] 10.222.0.83 () {56 vars in 1180 bytes} [Mon Jul 18 14:52:54 2022] GET /identity/v3/groups/58aaffd186e74fb8a3cf1f4124a157e1 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:55.108271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8815be7-53fd-46ea-a55e-eba12b36bb3c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:55.194081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:55.218243 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8815be7-53fd-46ea-a55e-eba12b36bb3c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:55.254989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f8a6a-1ba1-4f4d-a58b-19859b876187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:55.255749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f8a6a-1ba1-4f4d-a58b-19859b876187 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:55.256038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f8a6a-1ba1-4f4d-a58b-19859b876187 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:55.256323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f8a6a-1ba1-4f4d-a58b-19859b876187 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:55.257156 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f8a6a-1ba1-4f4d-a58b-19859b876187 None admin] RBAC: Adding query filter params (group.id=58aaffd186e74fb8a3cf1f4124a157e1, role.id=a82e24db1e3c4f868a586e122dae393e, scope.project.id=cc9a9fe967674703a9776dca128bc387) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:55.257471 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f8a6a-1ba1-4f4d-a58b-19859b876187 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:55.258966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f8a6a-1ba1-4f4d-a58b-19859b876187 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:55.265496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 230/455] 10.222.0.83 () {56 vars in 1376 bytes} [Mon Jul 18 14:52:55 2022] GET /identity/v3/role_assignments?group.id=58aaffd186e74fb8a3cf1f4124a157e1&scope.project.id=cc9a9fe967674703a9776dca128bc387&role.id=a82e24db1e3c4f868a586e122dae393e => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:58.149281 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac7a59-3b50-4eab-847e-3bc72093dd83 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:58.150344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac7a59-3b50-4eab-847e-3bc72093dd83 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:58.150874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac7a59-3b50-4eab-847e-3bc72093dd83 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:58.153759 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 226/456] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:58.160004 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012116b-437a-4814-947e-b8586d9c7072 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:58.160529 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012116b-437a-4814-947e-b8586d9c7072 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:58.160608 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012116b-437a-4814-947e-b8586d9c7072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:58.184271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-e012116b-437a-4814-947e-b8586d9c7072 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:58.185415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e012116b-437a-4814-947e-b8586d9c7072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:58.204485 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d5243-e636-43fe-8efd-c4c3f999cb18 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:58.205145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d5243-e636-43fe-8efd-c4c3f999cb18 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:58.205633 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d5243-e636-43fe-8efd-c4c3f999cb18 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:58.207291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 227/457] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:58.213424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51033036-216c-4d6b-b717-3c3c717a22e4 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:58.213709 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51033036-216c-4d6b-b717-3c3c717a22e4 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:58.214204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51033036-216c-4d6b-b717-3c3c717a22e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:58.237346 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-51033036-216c-4d6b-b717-3c3c717a22e4 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:58.241099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-51033036-216c-4d6b-b717-3c3c717a22e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:58.304221 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 231/458] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:58.343400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:58.353342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012116b-437a-4814-947e-b8586d9c7072 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:58.355945 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 228/459] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:58.410138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51033036-216c-4d6b-b717-3c3c717a22e4 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:58.459213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:58.523366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012116b-437a-4814-947e-b8586d9c7072 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:58.575403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51033036-216c-4d6b-b717-3c3c717a22e4 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:58.607396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710850a-da59-43b0-babe-32deec7aa360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:58.608992 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710850a-da59-43b0-babe-32deec7aa360 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:58.611259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710850a-da59-43b0-babe-32deec7aa360 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:58.612748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710850a-da59-43b0-babe-32deec7aa360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:58.634762 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710850a-da59-43b0-babe-32deec7aa360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:58.641905 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710850a-da59-43b0-babe-32deec7aa360 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:58.643620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263830f8-ae4b-442c-9d0e-f53b7806182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:58.644672 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263830f8-ae4b-442c-9d0e-f53b7806182f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:58.645107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263830f8-ae4b-442c-9d0e-f53b7806182f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:58.645567 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263830f8-ae4b-442c-9d0e-f53b7806182f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:58.666487 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263830f8-ae4b-442c-9d0e-f53b7806182f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:58.669131 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263830f8-ae4b-442c-9d0e-f53b7806182f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:58.686171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-f710850a-da59-43b0-babe-32deec7aa360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:52:58.686171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:58.686171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:58.686171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:58.686171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:58.686171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:58.686171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:58.686171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:58.686171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:58.686171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:58.686171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:58.686171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:58.686171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:52:58.686171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:52:58.686171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:58.687219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:58.688091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:52:58.688091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:52:58.688091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:52:58.688091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:58.688091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 232/460] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:52:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:58.688974 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad62095-75ed-4ce3-8320-27f594fd3c7d None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:58.689290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad62095-75ed-4ce3-8320-27f594fd3c7d None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:58.689632 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad62095-75ed-4ce3-8320-27f594fd3c7d None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:58.691190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 233/461] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:58.691863 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad62095-75ed-4ce3-8320-27f594fd3c7d None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:58.706714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-263830f8-ae4b-442c-9d0e-f53b7806182f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:52:58.706714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:58.706714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:58.706714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:58.706714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:58.706714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:58.706714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:58.706714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:58.706714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:58.706714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:58.706714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:58.706714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:58.706714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:52:58.706714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:52:58.706714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:58.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:58.709062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:52:58.709062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:52:58.709062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:52:58.709062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:52:58.709062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 229/462] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:52:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:52:58.709938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f214345-bc71-46ea-ba18-08d12f36568c None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:58.710923 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f214345-bc71-46ea-ba18-08d12f36568c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:58.717744 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f214345-bc71-46ea-ba18-08d12f36568c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:58.739061 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-2f214345-bc71-46ea-ba18-08d12f36568c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:58.740345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-2f214345-bc71-46ea-ba18-08d12f36568c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:58.813026 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:58.840844 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad62095-75ed-4ce3-8320-27f594fd3c7d None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:58.842798 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 230/463] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:58.843942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f214345-bc71-46ea-ba18-08d12f36568c None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:58.891627 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:58.904690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f8112-5fd8-4c1a-9b67-f4a622f6af7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:58.905375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f8112-5fd8-4c1a-9b67-f4a622f6af7a None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:58.905793 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f8112-5fd8-4c1a-9b67-f4a622f6af7a None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:58.906359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f8112-5fd8-4c1a-9b67-f4a622f6af7a None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:58.906829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f8112-5fd8-4c1a-9b67-f4a622f6af7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:58.907166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f8112-5fd8-4c1a-9b67-f4a622f6af7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:58.908647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f8112-5fd8-4c1a-9b67-f4a622f6af7a None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:58.922719 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 234/464] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:58.922719 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f8112-5fd8-4c1a-9b67-f4a622f6af7a None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:58.965578 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f214345-bc71-46ea-ba18-08d12f36568c None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:59.006382 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b13234-b16c-4be8-bc26-5a3d2230f90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:59.007111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b13234-b16c-4be8-bc26-5a3d2230f90e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:59.007392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b13234-b16c-4be8-bc26-5a3d2230f90e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:59.007666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b13234-b16c-4be8-bc26-5a3d2230f90e None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:59.008390 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b13234-b16c-4be8-bc26-5a3d2230f90e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:59.008696 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b13234-b16c-4be8-bc26-5a3d2230f90e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:59.010143 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b13234-b16c-4be8-bc26-5a3d2230f90e None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:59.029934 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 231/465] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:59.031414 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0f438-322d-4b93-ab4a-60837860d637 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:59.031709 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0f438-322d-4b93-ab4a-60837860d637 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:59.031988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0f438-322d-4b93-ab4a-60837860d637 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:59.033422 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 232/466] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:59.033983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0f438-322d-4b93-ab4a-60837860d637 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:59.052837 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f8112-5fd8-4c1a-9b67-f4a622f6af7a None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:59.086030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:59.089244 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83deaef6-9706-40b9-b8e6-7aec1d722cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:59.090248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83deaef6-9706-40b9-b8e6-7aec1d722cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:59.090564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83deaef6-9706-40b9-b8e6-7aec1d722cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:59.090859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83deaef6-9706-40b9-b8e6-7aec1d722cf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:59.102758 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83deaef6-9706-40b9-b8e6-7aec1d722cf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:59.104291 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83deaef6-9706-40b9-b8e6-7aec1d722cf6 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:59.124716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-83deaef6-9706-40b9-b8e6-7aec1d722cf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:52:59.124716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:59.124716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:59.124716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:59.124716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:59.124716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:59.124716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:59.124716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:59.124716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:59.124716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:59.124716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:59.124716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:59.124716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:52:59.124716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:52:59.124716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:59.125677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:59.131579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:52:59.131579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:52:59.131579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:52:59.131579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:59.131579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 235/467] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:52:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:59.131579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83deaef6-9706-40b9-b8e6-7aec1d722cf6 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:59.158482 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0f438-322d-4b93-ab4a-60837860d637 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:59.172276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:59.196430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31a181-6184-48b2-911b-1c8fe1bfabff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:59.197116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31a181-6184-48b2-911b-1c8fe1bfabff None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:59.197407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31a181-6184-48b2-911b-1c8fe1bfabff None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:59.200637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31a181-6184-48b2-911b-1c8fe1bfabff None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:59.200637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31a181-6184-48b2-911b-1c8fe1bfabff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:59.200637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31a181-6184-48b2-911b-1c8fe1bfabff None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:59.250043 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83deaef6-9706-40b9-b8e6-7aec1d722cf6 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:59.285215 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37689ac-703c-471a-8cce-54f5e268fb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:59.285888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37689ac-703c-471a-8cce-54f5e268fb96 None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:59.286179 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37689ac-703c-471a-8cce-54f5e268fb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:59.287171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37689ac-703c-471a-8cce-54f5e268fb96 None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:59.287305 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37689ac-703c-471a-8cce-54f5e268fb96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:59.288693 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37689ac-703c-471a-8cce-54f5e268fb96 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:59.298676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 233/468] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:52:59 2022] POST /identity/v3/users => generated 246 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:52:59.300303 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561b88f-e7e6-4bbc-b575-4c7ef2b04309 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:59.300594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561b88f-e7e6-4bbc-b575-4c7ef2b04309 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:59.300874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561b88f-e7e6-4bbc-b575-4c7ef2b04309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:59.325953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-6561b88f-e7e6-4bbc-b575-4c7ef2b04309 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:52:59.327290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-6561b88f-e7e6-4bbc-b575-4c7ef2b04309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:52:59.419734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 234/469] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:52:59.421167 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561b88f-e7e6-4bbc-b575-4c7ef2b04309 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:59.452186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 236/470] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:52:59 2022] POST /identity/v3/users => generated 248 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:52:59.459463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37689ac-703c-471a-8cce-54f5e268fb96 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:59.493291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:59.602492 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:59.616193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37689ac-703c-471a-8cce-54f5e268fb96 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:59.618427 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561b88f-e7e6-4bbc-b575-4c7ef2b04309 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:59.668408 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640676f6-efb6-43aa-9a5d-98a85298ea6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:59.670710 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640676f6-efb6-43aa-9a5d-98a85298ea6b None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:59.671394 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640676f6-efb6-43aa-9a5d-98a85298ea6b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:59.671897 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640676f6-efb6-43aa-9a5d-98a85298ea6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:59.685874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20850109-7373-4fa1-b017-b1ac6826dac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:59.686585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20850109-7373-4fa1-b017-b1ac6826dac8 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:59.686876 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20850109-7373-4fa1-b017-b1ac6826dac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:59.687151 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20850109-7373-4fa1-b017-b1ac6826dac8 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:59.687894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20850109-7373-4fa1-b017-b1ac6826dac8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:59.688197 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20850109-7373-4fa1-b017-b1ac6826dac8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:59.691604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20850109-7373-4fa1-b017-b1ac6826dac8 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:59.697343 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640676f6-efb6-43aa-9a5d-98a85298ea6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:59.700566 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640676f6-efb6-43aa-9a5d-98a85298ea6b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:59.711268 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 235/471] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:52:59.719135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20850109-7373-4fa1-b017-b1ac6826dac8 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:59.729855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-640676f6-efb6-43aa-9a5d-98a85298ea6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:52:59.729855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:52:59.729855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:52:59.729855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:52:59.729855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:52:59.729855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:52:59.729855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:52:59.729855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:52:59.729855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:52:59.729855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:52:59.729855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:52:59.729855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:52:59.729855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:52:59.729855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:52:59.729855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:52:59.730902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:52:59.731787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:52:59.731787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:52:59.731787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:52:59.731787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:52:59.735220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 237/472] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:52:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:52:59.737249 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae7a2b-a8d8-441e-9c96-4897aff42221 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:59.737777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae7a2b-a8d8-441e-9c96-4897aff42221 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:59.738854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae7a2b-a8d8-441e-9c96-4897aff42221 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:59.741158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 238/473] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:52:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:52:59.742879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae7a2b-a8d8-441e-9c96-4897aff42221 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:52:59.851744 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:52:59.868047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20850109-7373-4fa1-b017-b1ac6826dac8 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:59.901560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae7a2b-a8d8-441e-9c96-4897aff42221 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:52:59.915521 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb233272-1bd0-4ccb-8590-67d6341b9b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:59.916669 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb233272-1bd0-4ccb-8590-67d6341b9b9c None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:59.917483 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb233272-1bd0-4ccb-8590-67d6341b9b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:59.918499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb233272-1bd0-4ccb-8590-67d6341b9b9c None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:59.919297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb233272-1bd0-4ccb-8590-67d6341b9b9c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:59.921641 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb233272-1bd0-4ccb-8590-67d6341b9b9c None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:59.956441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798f785-d2c9-48c5-8bee-7fbafb4936c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:52:59.957594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798f785-d2c9-48c5-8bee-7fbafb4936c4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:59.958104 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798f785-d2c9-48c5-8bee-7fbafb4936c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:59.958663 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798f785-d2c9-48c5-8bee-7fbafb4936c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:59.959668 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798f785-d2c9-48c5-8bee-7fbafb4936c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:52:59.960273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798f785-d2c9-48c5-8bee-7fbafb4936c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:52:59.962013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798f785-d2c9-48c5-8bee-7fbafb4936c4 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:52:59.978837 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 239/474] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:52:59.981177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f800d6e-5db9-4994-9bb0-e9378eec0948 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:52:59.981811 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f800d6e-5db9-4994-9bb0-e9378eec0948 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:52:59.982320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f800d6e-5db9-4994-9bb0-e9378eec0948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:52:59.990225 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:00.004206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-4f800d6e-5db9-4994-9bb0-e9378eec0948 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:00.007290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-4f800d6e-5db9-4994-9bb0-e9378eec0948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:00.033436 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 236/475] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:52:59 2022] POST /identity/v3/users => generated 248 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:53:00.038775 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb233272-1bd0-4ccb-8590-67d6341b9b9c None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:00.129209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 240/476] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:00.211679 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f800d6e-5db9-4994-9bb0-e9378eec0948 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:00.249017 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb233272-1bd0-4ccb-8590-67d6341b9b9c None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:00.303221 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d0234-830e-4850-9620-c9cbc2055d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:00.304032 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d0234-830e-4850-9620-c9cbc2055d81 None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:00.304387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d0234-830e-4850-9620-c9cbc2055d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:00.304734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d0234-830e-4850-9620-c9cbc2055d81 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:00.305570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d0234-830e-4850-9620-c9cbc2055d81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:00.307296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d0234-830e-4850-9620-c9cbc2055d81 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:00.362810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f800d6e-5db9-4994-9bb0-e9378eec0948 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:00.397399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:00.403075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:00.414807 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22e1c5-5f54-4f9a-8e26-985207647fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:00.414807 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22e1c5-5f54-4f9a-8e26-985207647fe7 None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:00.414807 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22e1c5-5f54-4f9a-8e26-985207647fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:00.415111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 237/477] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:53:00 2022] POST /identity/v3/users => generated 249 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:53:00.415188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22e1c5-5f54-4f9a-8e26-985207647fe7 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:00.415188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22e1c5-5f54-4f9a-8e26-985207647fe7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:00.415188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22e1c5-5f54-4f9a-8e26-985207647fe7 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:00.472369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.sql.core [None req-ec22e1c5-5f54-4f9a-8e26-985207647fe7 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 14:53:00.472369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 14:53:00.472369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [parameters: {'id': 'd3303293ffdf448a999c21b59182eda7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 14:53:00.472369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80304) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 14:53:00.490607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-ec22e1c5-5f54-4f9a-8e26-985207647fe7 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 14:53:00.490607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:00.490607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 14:53:00.490607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 14:53:00.490607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 14:53:00.490607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 14:53:00.490607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 14:53:00.490607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 14:53:00.490607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 14:53:00.490607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application conn.query(q) Jul 18 14:53:00.490607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 14:53:00.490607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 14:53:00.490607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 14:53:00.490607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application result.read() Jul 18 14:53:00.490607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return ref.to_dict() Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 14:53:00.491588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application next(self.gen) Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application yield resource Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application next(self.gen) Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise self.value Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application session.commit() Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 14:53:00.493367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self.session.flush() Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application compat.raise_( Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 14:53:00.496100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 14:53:00.502860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 14:53:00.502860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 14:53:00.502860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 14:53:00.502860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 14:53:00.502860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 14:53:00.502860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 14:53:00.502860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 14:53:00.502860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 14:53:00.502860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 14:53:00.502860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 14:53:00.502860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 14:53:00.502860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 14:53:00.502860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 14:53:00.502860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 14:53:00.502860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 14:53:00.502860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception Jul 18 14:53:00.505039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 14:53:00.505039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 14:53:00.505039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 14:53:00.505039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 14:53:00.505039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 14:53:00.505039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 14:53:00.505039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 14:53:00.505039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application conn.query(q) Jul 18 14:53:00.505039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 14:53:00.505039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 14:53:00.505039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 14:53:00.505039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application result.read() Jul 18 14:53:00.505039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 14:53:00.505039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 14:53:00.505039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 14:53:00.505039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 14:53:00.507157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 14:53:00.507157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 14:53:00.507157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 14:53:00.507157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 14:53:00.507157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 14:53:00.507157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 14:53:00.507157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [parameters: {'id': 'd3303293ffdf448a999c21b59182eda7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 14:53:00.507157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 14:53:00.507157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:00.507157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:00.507157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:00.507157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:00.507157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:00.507157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:00.507157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:00.507157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:00.509214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:00.509214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:00.509214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:00.509214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:00.509214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:00.509214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:00.509214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 18 14:53:00.509214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 18 14:53:00.509214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:00.509214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:00.509214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 18 14:53:00.509214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 18 14:53:00.509214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 14:53:00.509214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 14:53:00.509214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 14:53:00.509214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:00.511497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 241/478] 10.222.0.83 () {60 vars in 1131 bytes} [Mon Jul 18 14:53:00 2022] POST /identity/v3/roles => generated 144 bytes in 281 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:00.511643 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d0234-830e-4850-9620-c9cbc2055d81 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:00.672968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d0234-830e-4850-9620-c9cbc2055d81 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:00.717679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0440df8-d110-4470-bf59-56767099dab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:00.717679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0440df8-d110-4470-bf59-56767099dab6 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:00.717679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0440df8-d110-4470-bf59-56767099dab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:00.717679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0440df8-d110-4470-bf59-56767099dab6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:00.733165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0440df8-d110-4470-bf59-56767099dab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:00.744418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-c0440df8-d110-4470-bf59-56767099dab6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:53:00.744418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:00.744418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:00.744418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:00.744418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:00.744418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:00.744418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:00.744418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:00.744418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:00.744418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:00.744418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:00.744418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:00.744418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:00.744418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:00.744418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:00.744418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:00.745962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:00.745962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:00.745962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:00.745962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:00.745962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:00.745962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:00.745962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:00.745962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:00.745962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:00.745962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:00.745962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:00.745962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:00.745962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:00.745962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:00.745962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:00.745962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:00.746795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:00.746795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:00.746795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:00.746795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:00.746795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:00.746795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:00.746795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:00.746795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:00.746795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:00.746795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:00.746795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:53:00.746795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:00.749327 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 238/479] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:53:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:00.756881 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22e1c5-5f54-4f9a-8e26-985207647fe7 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:00.887540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22e1c5-5f54-4f9a-8e26-985207647fe7 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:00.938591 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf17a2-7ba6-44a6-a897-f47ba0636463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:00.939297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf17a2-7ba6-44a6-a897-f47ba0636463 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:00.939588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf17a2-7ba6-44a6-a897-f47ba0636463 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:00.939864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf17a2-7ba6-44a6-a897-f47ba0636463 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:00.940665 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf17a2-7ba6-44a6-a897-f47ba0636463 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:00.941011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf17a2-7ba6-44a6-a897-f47ba0636463 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:00.942463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf17a2-7ba6-44a6-a897-f47ba0636463 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:00.953530 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 242/480] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:01.796437 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd447ff2-f0cf-4d57-9f42-b3954cdee26a None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:01.797030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd447ff2-f0cf-4d57-9f42-b3954cdee26a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:01.797030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd447ff2-f0cf-4d57-9f42-b3954cdee26a None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:01.798663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 239/481] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:01.805237 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fff34f-07dd-4025-87de-b8f6891145dc None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:01.805777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fff34f-07dd-4025-87de-b8f6891145dc None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:01.806217 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fff34f-07dd-4025-87de-b8f6891145dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:01.844455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-86fff34f-07dd-4025-87de-b8f6891145dc None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:01.846584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-86fff34f-07dd-4025-87de-b8f6891145dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:01.939831 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5fbfd-46dd-425f-a383-32776bd47bdc None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:01.940178 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5fbfd-46dd-425f-a383-32776bd47bdc None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:01.940401 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5fbfd-46dd-425f-a383-32776bd47bdc None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:01.942031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 240/482] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:01.952282 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640481c1-fe24-4dc4-accc-dee32512be03 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:01.952584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640481c1-fe24-4dc4-accc-dee32512be03 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:01.952857 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640481c1-fe24-4dc4-accc-dee32512be03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:01.962771 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:01.993783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 243/483] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:02.003879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-640481c1-fe24-4dc4-accc-dee32512be03 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:02.005855 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-640481c1-fe24-4dc4-accc-dee32512be03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:02.018606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fff34f-07dd-4025-87de-b8f6891145dc None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:02.161865 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:02.191539 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 241/484] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:02.237919 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640481c1-fe24-4dc4-accc-dee32512be03 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:02.243349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fff34f-07dd-4025-87de-b8f6891145dc None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:02.287185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd6af2-c834-4114-b402-cee1e168b442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:02.291742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd6af2-c834-4114-b402-cee1e168b442 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:02.292630 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd6af2-c834-4114-b402-cee1e168b442 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:02.293273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd6af2-c834-4114-b402-cee1e168b442 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:02.317902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd6af2-c834-4114-b402-cee1e168b442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:02.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-bafd6af2-c834-4114-b402-cee1e168b442 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:02.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:02.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:02.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:02.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:02.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:02.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:02.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:02.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:02.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:02.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:02.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:02.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:02.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:02.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:02.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:02.333543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:02.333543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:02.333543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:02.333543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:02.333543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:02.333543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:02.333543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:02.333543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:02.333543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:02.333543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:02.333543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:02.333543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:02.333543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:02.333543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:02.333543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:02.333543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:02.334337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:02.334337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:02.334337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:02.334337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:02.334337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:02.334337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:02.334337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:02.334337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:02.334337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:02.334337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:02.334337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:02.334337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:02.344554 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 244/485] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:02.350590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd6af2-c834-4114-b402-cee1e168b442 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:02.460437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:02.488343 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640481c1-fe24-4dc4-accc-dee32512be03 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:02.533208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd6af2-c834-4114-b402-cee1e168b442 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:02.580579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6c942-f4c6-43d8-b055-591471d9cff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:02.581412 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6c942-f4c6-43d8-b055-591471d9cff9 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:02.581700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6c942-f4c6-43d8-b055-591471d9cff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:02.581980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6c942-f4c6-43d8-b055-591471d9cff9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:02.583200 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949f018-91a9-4692-af42-5df37661666d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:02.583879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949f018-91a9-4692-af42-5df37661666d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:02.584162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949f018-91a9-4692-af42-5df37661666d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:02.584447 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949f018-91a9-4692-af42-5df37661666d None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:02.585203 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949f018-91a9-4692-af42-5df37661666d None admin] RBAC: Adding query filter params (name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:02.585515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949f018-91a9-4692-af42-5df37661666d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:02.586965 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949f018-91a9-4692-af42-5df37661666d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:02.600344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 245/486] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:02.606951 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949f018-91a9-4692-af42-5df37661666d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:02.615024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6c942-f4c6-43d8-b055-591471d9cff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:02.625800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-7bd6c942-f4c6-43d8-b055-591471d9cff9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:02.625800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:02.625800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:02.625800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:02.625800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:02.625800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:02.625800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:02.625800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:02.625800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:02.625800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:02.625800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:02.625800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:02.625800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:02.625800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:02.625800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:02.625800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:02.626809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:02.626809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:02.626809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:02.626809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:02.626809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:02.626809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:02.626809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:02.626809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:02.626809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:02.626809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:02.626809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:02.626809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:02.626809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:02.626809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:02.626809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:02.626809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:02.627607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:02.627607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:02.627607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:02.627607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:02.627607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:02.627607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:02.627607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:02.627607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:02.627607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:02.627607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:02.627607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:02.627607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:02.630475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 242/487] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:02.640257 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6c942-f4c6-43d8-b055-591471d9cff9 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:02.739431 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:02.747039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949f018-91a9-4692-af42-5df37661666d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:02.758847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:02.785885 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2c8f6-a78b-4d7d-94d1-0f7b80f4da35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:02.787008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2c8f6-a78b-4d7d-94d1-0f7b80f4da35 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:02.787300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2c8f6-a78b-4d7d-94d1-0f7b80f4da35 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:02.787748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2c8f6-a78b-4d7d-94d1-0f7b80f4da35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:02.801884 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2c8f6-a78b-4d7d-94d1-0f7b80f4da35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:02.804336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2c8f6-a78b-4d7d-94d1-0f7b80f4da35 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:02.807837 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6c942-f4c6-43d8-b055-591471d9cff9 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:02.829912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-58f2c8f6-a78b-4d7d-94d1-0f7b80f4da35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:02.829912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:02.829912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:02.829912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:02.829912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:02.829912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:02.829912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:02.829912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:02.829912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:02.829912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:02.829912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:02.829912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:02.829912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:02.829912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:02.829912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:02.831213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:02.832688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:02.832688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:02.832688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:02.832688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:02.832688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 246/488] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:02.832964 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cff12-3aff-4235-a985-84203f9b0da0 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:02.833194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cff12-3aff-4235-a985-84203f9b0da0 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:02.833457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cff12-3aff-4235-a985-84203f9b0da0 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:02.839396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 247/489] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:02.839396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01cff12-3aff-4235-a985-84203f9b0da0 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:02.856874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d5dcd-e447-4bea-aa42-1a9bd8734029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:02.857553 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d5dcd-e447-4bea-aa42-1a9bd8734029 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:02.857839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d5dcd-e447-4bea-aa42-1a9bd8734029 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:02.858121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d5dcd-e447-4bea-aa42-1a9bd8734029 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:02.859256 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d5dcd-e447-4bea-aa42-1a9bd8734029 None admin] RBAC: Adding query filter params (name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:02.859510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d5dcd-e447-4bea-aa42-1a9bd8734029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:02.860924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d5dcd-e447-4bea-aa42-1a9bd8734029 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:02.875094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 243/490] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:02.876617 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07725b9-d3e1-4e40-8049-8120728f77c4 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:02.876900 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07725b9-d3e1-4e40-8049-8120728f77c4 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:02.877193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07725b9-d3e1-4e40-8049-8120728f77c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:02.895637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-d07725b9-d3e1-4e40-8049-8120728f77c4 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:02.896844 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d07725b9-d3e1-4e40-8049-8120728f77c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:02.959347 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01cff12-3aff-4235-a985-84203f9b0da0 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:02.972584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 244/491] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:02.973691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07725b9-d3e1-4e40-8049-8120728f77c4 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:03.025071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badad32-36c7-4f8e-8b91-3cdbd838f6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:03.026930 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badad32-36c7-4f8e-8b91-3cdbd838f6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:03.028523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badad32-36c7-4f8e-8b91-3cdbd838f6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:03.028985 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badad32-36c7-4f8e-8b91-3cdbd838f6c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:03.029880 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8badad32-36c7-4f8e-8b91-3cdbd838f6c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:03.030341 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8badad32-36c7-4f8e-8b91-3cdbd838f6c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:03.031921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8badad32-36c7-4f8e-8b91-3cdbd838f6c0 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:03.037649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:03.050662 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 248/492] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:03.051996 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badad32-36c7-4f8e-8b91-3cdbd838f6c0 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:03.086838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:03.104824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07725b9-d3e1-4e40-8049-8120728f77c4 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:03.143850 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b96d95-f544-49f4-b3db-ba596b7026d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:03.144649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b96d95-f544-49f4-b3db-ba596b7026d9 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:03.144957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b96d95-f544-49f4-b3db-ba596b7026d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:03.145231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b96d95-f544-49f4-b3db-ba596b7026d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:03.156616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b96d95-f544-49f4-b3db-ba596b7026d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:03.158097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b96d95-f544-49f4-b3db-ba596b7026d9 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:03.176788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-c6b96d95-f544-49f4-b3db-ba596b7026d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:03.176788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:03.176788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:03.176788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:03.176788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:03.176788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:03.176788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:03.176788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:03.176788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:03.176788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:03.176788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:03.176788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:03.176788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:03.176788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:03.176788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:03.177735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:03.179309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:03.179309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:03.179309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:03.179309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:03.179309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 245/493] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:03.179309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b96d95-f544-49f4-b3db-ba596b7026d9 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:03.185849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badad32-36c7-4f8e-8b91-3cdbd838f6c0 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:03.225538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff049d-32f6-44d2-8136-c3a7a86f4586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:03.226403 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff049d-32f6-44d2-8136-c3a7a86f4586 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:03.226763 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff049d-32f6-44d2-8136-c3a7a86f4586 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:03.227049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff049d-32f6-44d2-8136-c3a7a86f4586 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:03.239810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff049d-32f6-44d2-8136-c3a7a86f4586 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:03.248196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-f4ff049d-32f6-44d2-8136-c3a7a86f4586 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:03.248196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:03.248196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:03.248196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:03.248196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:03.248196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:03.248196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:03.248196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:03.248196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:03.248196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:03.248196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:03.248196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:03.248196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:03.248196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:03.248196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:03.248196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:03.249294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:03.249294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:03.249294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:03.249294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:03.249294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:03.249294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:03.249294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:03.249294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:03.249294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:03.249294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:03.249294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:03.249294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:03.249294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:03.249294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:03.249294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:03.249294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:03.250205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:03.250205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:03.250205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:03.250205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:03.250205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:03.250205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:03.250205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:03.250205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:03.250205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:03.250205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:03.250205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:03.250205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:03.251288 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 249/494] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:03.251775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb39f7-1dbc-479d-8a8c-2517f425d2bc None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:03.252070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb39f7-1dbc-479d-8a8c-2517f425d2bc None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:03.252355 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb39f7-1dbc-479d-8a8c-2517f425d2bc None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:03.253788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 250/495] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:03.258263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb39f7-1dbc-479d-8a8c-2517f425d2bc None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:03.293598 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:03.302315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b96d95-f544-49f4-b3db-ba596b7026d9 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:03.345642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c199ee-e441-4b78-98f4-bf337380c3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:03.346663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c199ee-e441-4b78-98f4-bf337380c3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:03.346893 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c199ee-e441-4b78-98f4-bf337380c3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:03.347170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c199ee-e441-4b78-98f4-bf337380c3c4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:03.350156 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:03.357659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c199ee-e441-4b78-98f4-bf337380c3c4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:03.360964 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c199ee-e441-4b78-98f4-bf337380c3c4 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:03.370524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb39f7-1dbc-479d-8a8c-2517f425d2bc None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:03.377877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-a9c199ee-e441-4b78-98f4-bf337380c3c4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:53:03.377877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:03.377877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:03.377877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:03.377877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:03.377877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:03.377877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:03.377877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:03.377877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:03.377877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:03.377877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:03.377877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:03.377877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:03.377877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:03.377877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:03.377877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:03.378830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:03.380160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:03.380160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:03.380160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:03.380160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:03.380160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:03.380160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:03.380160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:03.380160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:03.380160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:53:03.380160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:03.382880 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 246/496] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Jul 18 14:53:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:03.383972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c199ee-e441-4b78-98f4-bf337380c3c4 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:03.410158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d1ecf-838c-44dd-9337-90d386ef1370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:03.410871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d1ecf-838c-44dd-9337-90d386ef1370 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:03.411165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d1ecf-838c-44dd-9337-90d386ef1370 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:03.411449 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d1ecf-838c-44dd-9337-90d386ef1370 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:03.412280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d1ecf-838c-44dd-9337-90d386ef1370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:03.412591 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d1ecf-838c-44dd-9337-90d386ef1370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:03.413996 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d1ecf-838c-44dd-9337-90d386ef1370 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:03.426656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 251/497] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:03.427929 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6106f06-f21c-4670-9725-5d264b6e3431 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:03.428229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6106f06-f21c-4670-9725-5d264b6e3431 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:03.428513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6106f06-f21c-4670-9725-5d264b6e3431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:03.449271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-c6106f06-f21c-4670-9725-5d264b6e3431 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:03.450562 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c6106f06-f21c-4670-9725-5d264b6e3431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:03.508206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c199ee-e441-4b78-98f4-bf337380c3c4 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:03.538585 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 252/498] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:03.538585 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6106f06-f21c-4670-9725-5d264b6e3431 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:03.560385 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691b727-5115-475b-b4ed-3447b29dc179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:03.561308 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691b727-5115-475b-b4ed-3447b29dc179 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:03.561609 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691b727-5115-475b-b4ed-3447b29dc179 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:03.562156 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691b727-5115-475b-b4ed-3447b29dc179 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:03.562948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691b727-5115-475b-b4ed-3447b29dc179 None admin] RBAC: Adding query filter params (name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:03.563265 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691b727-5115-475b-b4ed-3447b29dc179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:03.565645 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691b727-5115-475b-b4ed-3447b29dc179 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:03.580300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 247/499] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:03.580300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691b727-5115-475b-b4ed-3447b29dc179 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:03.597659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:03.617889 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:03.648395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6106f06-f21c-4670-9725-5d264b6e3431 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:03.684076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691b727-5115-475b-b4ed-3447b29dc179 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:03.685953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6eece-594f-4c6b-b31c-1edfe8044ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:03.686664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6eece-594f-4c6b-b31c-1edfe8044ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:03.686953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6eece-594f-4c6b-b31c-1edfe8044ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:03.687234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6eece-594f-4c6b-b31c-1edfe8044ef0 None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:03.688077 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6eece-594f-4c6b-b31c-1edfe8044ef0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:03.688389 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6eece-594f-4c6b-b31c-1edfe8044ef0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:03.689830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6eece-594f-4c6b-b31c-1edfe8044ef0 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:03.721165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137821e8-f1ae-4f18-805b-b8f293303e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:03.721975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137821e8-f1ae-4f18-805b-b8f293303e7f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:03.722256 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137821e8-f1ae-4f18-805b-b8f293303e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:03.722560 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137821e8-f1ae-4f18-805b-b8f293303e7f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:03.734396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 253/500] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Jul 18 14:53:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:03.734396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6eece-594f-4c6b-b31c-1edfe8044ef0 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:03.736362 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137821e8-f1ae-4f18-805b-b8f293303e7f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:03.737998 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137821e8-f1ae-4f18-805b-b8f293303e7f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:03.753871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-137821e8-f1ae-4f18-805b-b8f293303e7f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 14:53:03.753871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:03.753871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:03.753871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:03.753871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:03.753871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:03.753871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:03.753871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:03.753871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:03.753871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:03.753871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:03.753871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:03.753871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:03.753871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:03.753871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:03.753871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:03.754948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:03.756725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:03.756725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:03.756725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:03.756725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:03.756725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:03.756725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:03.756725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:03.756725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:03.756725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 14:53:03.756725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:03.756725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 248/501] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:53:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:03.756725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137821e8-f1ae-4f18-805b-b8f293303e7f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:03.841798 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:03.846275 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6eece-594f-4c6b-b31c-1edfe8044ef0 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:03.874276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137821e8-f1ae-4f18-805b-b8f293303e7f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:03.879493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:03.887168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e0f54-08b5-496d-865c-568c5c9e75f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:03.888037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e0f54-08b5-496d-865c-568c5c9e75f1 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:03.888304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e0f54-08b5-496d-865c-568c5c9e75f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:03.888581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e0f54-08b5-496d-865c-568c5c9e75f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:03.901059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e0f54-08b5-496d-865c-568c5c9e75f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:03.910273 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5aeef-2e48-4a14-8524-53526f0fe3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:03.911100 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5aeef-2e48-4a14-8524-53526f0fe3ad None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:03.911299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-381e0f54-08b5-496d-865c-568c5c9e75f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:03.911299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:03.911299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:03.911299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:03.911299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:03.911299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:03.911299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:03.911299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:03.911299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:03.911299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:03.911299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:03.911299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:03.911299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:03.911299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:03.911299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:03.911299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:03.912206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:03.912206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:03.912206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:03.912206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:03.912206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:03.912206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:03.912206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:03.912206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:03.912206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:03.912206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:03.912206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:03.912206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:03.912206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:03.912206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:03.912206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:03.912206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:03.913026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:03.913026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:03.913026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:03.913026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:03.913026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:03.913026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:03.913026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:03.913026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:03.913026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:03.913026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:03.913026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:03.913026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:03.913655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5aeef-2e48-4a14-8524-53526f0fe3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:03.913655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5aeef-2e48-4a14-8524-53526f0fe3ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:03.913788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 254/502] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:03.913788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e0f54-08b5-496d-865c-568c5c9e75f1 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:03.926501 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d5aeef-2e48-4a14-8524-53526f0fe3ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:03.928026 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d5aeef-2e48-4a14-8524-53526f0fe3ad None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:03.946585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-93d5aeef-2e48-4a14-8524-53526f0fe3ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:03.946585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:03.946585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:03.946585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:03.946585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:03.946585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:03.946585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:03.946585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:03.946585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:03.946585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:03.946585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:03.946585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:03.946585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:03.946585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:03.946585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:03.947520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:03.954271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:03.954271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:03.954271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:03.954271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:03.954271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 249/503] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:03.954271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5aeef-2e48-4a14-8524-53526f0fe3ad None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:04.030660 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e0f54-08b5-496d-865c-568c5c9e75f1 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:04.058555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5aeef-2e48-4a14-8524-53526f0fe3ad None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:04.064267 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8dd80-ba51-48fe-bacb-0a03b4105df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:04.064917 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8dd80-ba51-48fe-bacb-0a03b4105df6 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:04.065209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8dd80-ba51-48fe-bacb-0a03b4105df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:04.065488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8dd80-ba51-48fe-bacb-0a03b4105df6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:04.092698 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:04.100675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f992c90-789c-4204-9d3a-4a09757857d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:04.101248 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f992c90-789c-4204-9d3a-4a09757857d8 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:04.101535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f992c90-789c-4204-9d3a-4a09757857d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:04.101807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f992c90-789c-4204-9d3a-4a09757857d8 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:04.102671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f992c90-789c-4204-9d3a-4a09757857d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:04.102981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f992c90-789c-4204-9d3a-4a09757857d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:04.105405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f992c90-789c-4204-9d3a-4a09757857d8 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:04.125099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:04.135185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8dd80-ba51-48fe-bacb-0a03b4105df6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:04.136784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8dd80-ba51-48fe-bacb-0a03b4105df6 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:04.142819 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 250/504] 10.222.0.83 () {56 vars in 1137 bytes} [Mon Jul 18 14:53:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:04.143665 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f992c90-789c-4204-9d3a-4a09757857d8 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:04.243154 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 255/505] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:53:04.243154 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8dd80-ba51-48fe-bacb-0a03b4105df6 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:04.247303 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f992c90-789c-4204-9d3a-4a09757857d8 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:04.281267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77eda0-03fe-479d-973e-4997721775bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:04.281929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77eda0-03fe-479d-973e-4997721775bf None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:04.282220 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77eda0-03fe-479d-973e-4997721775bf None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:04.282500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77eda0-03fe-479d-973e-4997721775bf None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:04.283271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77eda0-03fe-479d-973e-4997721775bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:04.283575 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77eda0-03fe-479d-973e-4997721775bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:04.284991 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77eda0-03fe-479d-973e-4997721775bf None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:04.294100 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 251/506] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:04.295639 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17aa3a-a156-499d-882e-9c962d8da30d None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:04.295937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17aa3a-a156-499d-882e-9c962d8da30d None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:04.296207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17aa3a-a156-499d-882e-9c962d8da30d None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:04.297604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 252/507] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:04.298225 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17aa3a-a156-499d-882e-9c962d8da30d None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:04.332500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:04.342935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8dd80-ba51-48fe-bacb-0a03b4105df6 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:04.352749 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:04.378315 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8295a06-7df1-4e87-b355-53b4a8ff5ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:04.379011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8295a06-7df1-4e87-b355-53b4a8ff5ebf None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:04.379300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8295a06-7df1-4e87-b355-53b4a8ff5ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:04.379581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8295a06-7df1-4e87-b355-53b4a8ff5ebf None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:04.380307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8295a06-7df1-4e87-b355-53b4a8ff5ebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:04.380657 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8295a06-7df1-4e87-b355-53b4a8ff5ebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:04.382377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8295a06-7df1-4e87-b355-53b4a8ff5ebf None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:04.395705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 256/508] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:04.396597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8295a06-7df1-4e87-b355-53b4a8ff5ebf None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:04.414177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17aa3a-a156-499d-882e-9c962d8da30d None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:04.450205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe4b7f-a2c0-46eb-8ee8-b66f3c9d2d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:04.450881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe4b7f-a2c0-46eb-8ee8-b66f3c9d2d80 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:04.451169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe4b7f-a2c0-46eb-8ee8-b66f3c9d2d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:04.451445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe4b7f-a2c0-46eb-8ee8-b66f3c9d2d80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:04.501036 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8295a06-7df1-4e87-b355-53b4a8ff5ebf None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:04.521053 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe4b7f-a2c0-46eb-8ee8-b66f3c9d2d80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:04.522584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe4b7f-a2c0-46eb-8ee8-b66f3c9d2d80 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:04.535094 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e729a-c986-4550-a2d3-a4ac1c6fcb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:04.535908 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e729a-c986-4550-a2d3-a4ac1c6fcb62 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:04.536190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e729a-c986-4550-a2d3-a4ac1c6fcb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:04.536463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e729a-c986-4550-a2d3-a4ac1c6fcb62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:04.548172 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e729a-c986-4550-a2d3-a4ac1c6fcb62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:04.549628 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e729a-c986-4550-a2d3-a4ac1c6fcb62 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:04.567145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-004e729a-c986-4550-a2d3-a4ac1c6fcb62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:04.567145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:04.567145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:04.567145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:04.567145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:04.567145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:04.567145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:04.567145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:04.567145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:04.567145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:04.567145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:04.567145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:04.567145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:04.567145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:04.567145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:04.568039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:04.568912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:04.568912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:04.568912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:04.568912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:04.568912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 257/509] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:04.569180 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e729a-c986-4550-a2d3-a4ac1c6fcb62 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:04.575942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:04.576104 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:04.631006 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 253/510] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:53:04.632317 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eb537-d06a-4c2f-a176-18cd523c1391 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:04.632613 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eb537-d06a-4c2f-a176-18cd523c1391 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:04.632895 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eb537-d06a-4c2f-a176-18cd523c1391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:04.649550 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-468eb537-d06a-4c2f-a176-18cd523c1391 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:04.650816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-468eb537-d06a-4c2f-a176-18cd523c1391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:04.676151 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e729a-c986-4550-a2d3-a4ac1c6fcb62 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:04.710263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830551c-8a3c-4b38-85e6-2d43a065cfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:04.711097 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830551c-8a3c-4b38-85e6-2d43a065cfc0 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:04.711382 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830551c-8a3c-4b38-85e6-2d43a065cfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:04.711953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830551c-8a3c-4b38-85e6-2d43a065cfc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:04.719066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 254/511] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:04.731210 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468eb537-d06a-4c2f-a176-18cd523c1391 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:04.731322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830551c-8a3c-4b38-85e6-2d43a065cfc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:04.731322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830551c-8a3c-4b38-85e6-2d43a065cfc0 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:04.743622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-9830551c-8a3c-4b38-85e6-2d43a065cfc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:04.743622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:04.743622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:04.743622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:04.743622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:04.743622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:04.743622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:04.743622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:04.743622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:04.743622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:04.743622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:04.743622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:04.743622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:04.743622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:04.743622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:04.744558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:04.746040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:04.746040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:04.746040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:04.746040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:04.746040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 258/512] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:04.746750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830551c-8a3c-4b38-85e6-2d43a065cfc0 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:04.810042 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:04.819643 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:04.847931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468eb537-d06a-4c2f-a176-18cd523c1391 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:04.864361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830551c-8a3c-4b38-85e6-2d43a065cfc0 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:04.887154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9648de-6c8b-4cda-aec7-d1ea7c94e5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:04.887896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9648de-6c8b-4cda-aec7-d1ea7c94e5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:04.888184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9648de-6c8b-4cda-aec7-d1ea7c94e5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:04.888459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9648de-6c8b-4cda-aec7-d1ea7c94e5d1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:04.896753 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9648de-6c8b-4cda-aec7-d1ea7c94e5d1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:04.898290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9648de-6c8b-4cda-aec7-d1ea7c94e5d1 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:04.905407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a423b3-227d-48d3-992c-941500aa2b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:04.906168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a423b3-227d-48d3-992c-941500aa2b6c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:04.906558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a423b3-227d-48d3-992c-941500aa2b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:04.906924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a423b3-227d-48d3-992c-941500aa2b6c None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:04.907592 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a423b3-227d-48d3-992c-941500aa2b6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:04.907898 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a423b3-227d-48d3-992c-941500aa2b6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:04.909351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a423b3-227d-48d3-992c-941500aa2b6c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:04.914378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-7c9648de-6c8b-4cda-aec7-d1ea7c94e5d1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 14:53:04.914378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:04.914378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:04.914378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:04.914378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:04.914378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:04.914378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:04.914378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:04.914378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:04.914378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:04.914378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:04.914378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:04.914378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:04.914378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:04.914378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:04.914378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:04.915498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:04.917126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:04.917126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:04.917126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:04.917126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:04.917126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:04.917126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:04.917126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:04.917126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:04.917126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 14:53:04.917126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:04.917126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 255/513] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:53:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:04.917126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9648de-6c8b-4cda-aec7-d1ea7c94e5d1 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:04.932261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 259/514] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:04.935535 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a423b3-227d-48d3-992c-941500aa2b6c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:05.038182 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9648de-6c8b-4cda-aec7-d1ea7c94e5d1 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:05.049540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a423b3-227d-48d3-992c-941500aa2b6c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:05.065852 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:05.077761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a4b9e-551e-4fdf-8d8d-03515ecfd848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:05.078595 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a4b9e-551e-4fdf-8d8d-03515ecfd848 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:05.078889 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a4b9e-551e-4fdf-8d8d-03515ecfd848 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:05.079170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a4b9e-551e-4fdf-8d8d-03515ecfd848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:05.086342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:05.090221 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13daf795-b884-45d5-98c3-97afa1d955de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:05.091068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daf795-b884-45d5-98c3-97afa1d955de None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:05.091352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daf795-b884-45d5-98c3-97afa1d955de None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:05.091625 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daf795-b884-45d5-98c3-97afa1d955de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:05.095455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a4b9e-551e-4fdf-8d8d-03515ecfd848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:05.097139 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a4b9e-551e-4fdf-8d8d-03515ecfd848 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:05.104683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13daf795-b884-45d5-98c3-97afa1d955de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:05.106139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13daf795-b884-45d5-98c3-97afa1d955de None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:05.116699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-6f4a4b9e-551e-4fdf-8d8d-03515ecfd848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:05.116699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:05.116699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:05.116699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:05.116699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:05.116699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:05.116699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:05.116699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:05.116699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:05.116699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:05.116699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:05.116699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:05.116699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:05.116699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:05.116699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:05.117791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:05.119208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:05.119208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:05.119208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:05.119208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:05.119208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 256/515] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:05.119208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a4b9e-551e-4fdf-8d8d-03515ecfd848 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:05.126019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-13daf795-b884-45d5-98c3-97afa1d955de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:05.126019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:05.126019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:05.126019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:05.126019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:05.126019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:05.126019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:05.126019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:05.126019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:05.126019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:05.126019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:05.126019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:05.126019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:05.126019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:05.126019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:05.134052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:05.134993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:05.134993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:05.134993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:05.134993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:05.134993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 260/516] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:05.134993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13daf795-b884-45d5-98c3-97afa1d955de None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:05.231599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a4b9e-551e-4fdf-8d8d-03515ecfd848 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:05.249837 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13daf795-b884-45d5-98c3-97afa1d955de None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:05.273775 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60af13-6603-4192-b6eb-3787f4c849ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:05.274447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60af13-6603-4192-b6eb-3787f4c849ac None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:05.274764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60af13-6603-4192-b6eb-3787f4c849ac None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:05.275043 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60af13-6603-4192-b6eb-3787f4c849ac None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:05.275774 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60af13-6603-4192-b6eb-3787f4c849ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:05.276612 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60af13-6603-4192-b6eb-3787f4c849ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:05.278236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60af13-6603-4192-b6eb-3787f4c849ac None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:05.292829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd314687-f409-4d5c-ac32-179226774033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:05.293474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd314687-f409-4d5c-ac32-179226774033 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:05.293762 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd314687-f409-4d5c-ac32-179226774033 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:05.294039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd314687-f409-4d5c-ac32-179226774033 None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:05.294933 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd314687-f409-4d5c-ac32-179226774033 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:05.301912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd314687-f409-4d5c-ac32-179226774033 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:05.302189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 257/517] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:05.302189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60af13-6603-4192-b6eb-3787f4c849ac None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:05.302488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd314687-f409-4d5c-ac32-179226774033 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:05.322397 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 261/518] 10.222.0.83 () {56 vars in 1137 bytes} [Mon Jul 18 14:53:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:05.323837 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd314687-f409-4d5c-ac32-179226774033 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:05.352414 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:05.362029 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:05.421956 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60af13-6603-4192-b6eb-3787f4c849ac None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:05.442311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd314687-f409-4d5c-ac32-179226774033 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:05.472859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606e7d6-93b8-4163-a289-f1630bb5d05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:05.473781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606e7d6-93b8-4163-a289-f1630bb5d05e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:05.474139 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606e7d6-93b8-4163-a289-f1630bb5d05e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:05.474422 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606e7d6-93b8-4163-a289-f1630bb5d05e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:05.484132 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606e7d6-93b8-4163-a289-f1630bb5d05e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:05.485664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606e7d6-93b8-4163-a289-f1630bb5d05e None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:05.489782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f5da9-73e3-4f76-a47f-6c1567649bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:05.490469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f5da9-73e3-4f76-a47f-6c1567649bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:05.490791 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f5da9-73e3-4f76-a47f-6c1567649bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:05.491075 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f5da9-73e3-4f76-a47f-6c1567649bd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:05.491824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f5da9-73e3-4f76-a47f-6c1567649bd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:05.492291 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f5da9-73e3-4f76-a47f-6c1567649bd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:05.493583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f5da9-73e3-4f76-a47f-6c1567649bd5 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:05.502700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-1606e7d6-93b8-4163-a289-f1630bb5d05e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:05.502700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:05.502700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:05.502700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:05.502700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:05.502700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:05.502700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:05.502700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:05.502700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:05.502700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:05.502700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:05.502700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:05.502700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:05.502700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:05.502700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:05.510241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:05.511222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:05.511222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:05.511222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:05.511222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:05.511222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:05.511222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:05.511222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:05.511222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:05.511222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 258/519] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:05.511222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606e7d6-93b8-4163-a289-f1630bb5d05e None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:05.511922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 262/520] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:05.511922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f5da9-73e3-4f76-a47f-6c1567649bd5 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:05.613912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:05.630300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606e7d6-93b8-4163-a289-f1630bb5d05e None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:05.632842 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f5da9-73e3-4f76-a47f-6c1567649bd5 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:05.642556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:05.673363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc500864-ce42-4337-a3db-6dcdcd17a22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:05.673885 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387afe9-1daf-4729-8ee3-9a6f16ec4056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:05.674104 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc500864-ce42-4337-a3db-6dcdcd17a22c None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:05.674322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc500864-ce42-4337-a3db-6dcdcd17a22c None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:05.674646 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc500864-ce42-4337-a3db-6dcdcd17a22c None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:05.674814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387afe9-1daf-4729-8ee3-9a6f16ec4056 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:05.674995 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387afe9-1daf-4729-8ee3-9a6f16ec4056 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:05.675270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387afe9-1daf-4729-8ee3-9a6f16ec4056 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:05.675371 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc500864-ce42-4337-a3db-6dcdcd17a22c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:05.675691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc500864-ce42-4337-a3db-6dcdcd17a22c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:05.677122 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc500864-ce42-4337-a3db-6dcdcd17a22c None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:05.685113 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387afe9-1daf-4729-8ee3-9a6f16ec4056 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:05.686608 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387afe9-1daf-4729-8ee3-9a6f16ec4056 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:05.692208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 259/521] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:05.693091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc500864-ce42-4337-a3db-6dcdcd17a22c None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:05.704913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-a387afe9-1daf-4729-8ee3-9a6f16ec4056 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 14:53:05.704913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:05.704913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:05.704913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:05.704913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:05.704913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:05.704913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:05.704913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:05.704913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:05.704913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:05.704913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:05.704913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:05.704913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:05.704913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:05.704913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:05.704913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:05.706649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:05.708193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:05.708193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:05.708193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:05.708193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:05.708193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:05.708193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:05.708193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:05.708193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:05.708193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 14:53:05.708193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:05.708193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 263/522] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:05.708193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387afe9-1daf-4729-8ee3-9a6f16ec4056 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:05.820514 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc500864-ce42-4337-a3db-6dcdcd17a22c None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:05.833435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387afe9-1daf-4729-8ee3-9a6f16ec4056 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:05.871476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379c751-ab5b-4dfa-b2a0-d862cfb4e2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:05.872608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379c751-ab5b-4dfa-b2a0-d862cfb4e2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:05.873110 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379c751-ab5b-4dfa-b2a0-d862cfb4e2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:05.873627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379c751-ab5b-4dfa-b2a0-d862cfb4e2c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:05.879161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991c5e9-2f5c-44b1-a21e-434647541600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:05.879987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991c5e9-2f5c-44b1-a21e-434647541600 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:05.880149 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991c5e9-2f5c-44b1-a21e-434647541600 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:05.880369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991c5e9-2f5c-44b1-a21e-434647541600 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:05.881173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991c5e9-2f5c-44b1-a21e-434647541600 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:05.881481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991c5e9-2f5c-44b1-a21e-434647541600 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:05.882995 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991c5e9-2f5c-44b1-a21e-434647541600 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:05.904973 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 264/523] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:05.906477 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991c5e9-2f5c-44b1-a21e-434647541600 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:05.915737 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:05.936757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:05.971772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379c751-ab5b-4dfa-b2a0-d862cfb4e2c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:05.973995 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379c751-ab5b-4dfa-b2a0-d862cfb4e2c4 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:06.035558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991c5e9-2f5c-44b1-a21e-434647541600 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:06.080484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e5378-4368-4342-96af-c1ba7929bd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:06.081236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e5378-4368-4342-96af-c1ba7929bd42 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:06.081531 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e5378-4368-4342-96af-c1ba7929bd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:06.081970 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e5378-4368-4342-96af-c1ba7929bd42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:06.091199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e5378-4368-4342-96af-c1ba7929bd42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:06.092714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e5378-4368-4342-96af-c1ba7929bd42 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:06.096783 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 260/524] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:53:06.098169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379c751-ab5b-4dfa-b2a0-d862cfb4e2c4 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:06.111481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-ba4e5378-4368-4342-96af-c1ba7929bd42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:06.111481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:06.111481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:06.111481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:06.111481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:06.111481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:06.111481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:06.111481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:06.111481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:06.111481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:06.111481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:06.111481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:06.111481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:06.111481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:06.111481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:06.112580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:06.114127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:06.114127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:06.114127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:06.114127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:06.114127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:06.114127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:06.114127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:06.114127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:06.114127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 265/525] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:06.114127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e5378-4368-4342-96af-c1ba7929bd42 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:06.189880 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:06.193707 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:06.224940 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379c751-ab5b-4dfa-b2a0-d862cfb4e2c4 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:06.256648 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e5378-4368-4342-96af-c1ba7929bd42 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:06.263346 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d89a9-93ad-4e8f-932f-f81ec0ba9170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:06.264028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d89a9-93ad-4e8f-932f-f81ec0ba9170 None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:06.264310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d89a9-93ad-4e8f-932f-f81ec0ba9170 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:06.264586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d89a9-93ad-4e8f-932f-f81ec0ba9170 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:06.265297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d89a9-93ad-4e8f-932f-f81ec0ba9170 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:06.266821 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d89a9-93ad-4e8f-932f-f81ec0ba9170 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:06.308882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb45258-2e81-40a8-bc2a-fbc440be9e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:06.309559 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb45258-2e81-40a8-bc2a-fbc440be9e1d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:06.309842 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb45258-2e81-40a8-bc2a-fbc440be9e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:06.310124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb45258-2e81-40a8-bc2a-fbc440be9e1d None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:06.310999 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb45258-2e81-40a8-bc2a-fbc440be9e1d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:06.311341 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb45258-2e81-40a8-bc2a-fbc440be9e1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:06.312836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb45258-2e81-40a8-bc2a-fbc440be9e1d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:06.340970 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 261/526] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:53:06 2022] POST /identity/v3/users => generated 247 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:53:06.340970 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d89a9-93ad-4e8f-932f-f81ec0ba9170 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:06.342418 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 266/527] 10.222.0.83 () {56 vars in 1139 bytes} [Mon Jul 18 14:53:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:06.343862 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb45258-2e81-40a8-bc2a-fbc440be9e1d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:06.510162 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d89a9-93ad-4e8f-932f-f81ec0ba9170 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:06.513157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb45258-2e81-40a8-bc2a-fbc440be9e1d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:06.548573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:06.552203 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:06.560463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087a77a-240c-4c6c-9d6b-b04a6c081c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:06.563831 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087a77a-240c-4c6c-9d6b-b04a6c081c29 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:06.563831 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087a77a-240c-4c6c-9d6b-b04a6c081c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:06.563831 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087a77a-240c-4c6c-9d6b-b04a6c081c29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:06.564298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087a77a-240c-4c6c-9d6b-b04a6c081c29 None admin] RBAC: Adding query filter params (role.id=29b8b950372848ffa9a75e778ee66b4c, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=7e1355d18fb049468651b3675c217d60) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:06.564913 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087a77a-240c-4c6c-9d6b-b04a6c081c29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:06.567471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087a77a-240c-4c6c-9d6b-b04a6c081c29 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:06.579112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3613d-afc7-46d9-92b8-7c603c4dda98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:06.579461 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 262/528] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:06 2022] GET /identity/v3/role_assignments?user.id=7e1355d18fb049468651b3675c217d60&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=29b8b950372848ffa9a75e778ee66b4c => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:06.580770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3613d-afc7-46d9-92b8-7c603c4dda98 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:06.581359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087a77a-240c-4c6c-9d6b-b04a6c081c29 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:06.581502 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3613d-afc7-46d9-92b8-7c603c4dda98 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:06.581857 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3613d-afc7-46d9-92b8-7c603c4dda98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:06.599777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a3613d-afc7-46d9-92b8-7c603c4dda98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:06.601203 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a3613d-afc7-46d9-92b8-7c603c4dda98 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:06.628787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-39a3613d-afc7-46d9-92b8-7c603c4dda98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:06.628787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:06.628787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:06.628787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:06.628787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:06.628787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:06.628787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:06.628787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:06.628787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:06.628787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:06.628787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:06.628787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:06.628787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:06.628787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:06.628787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:06.630689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:06.633886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:06.633886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:06.633886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:06.633886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:06.637645 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 267/529] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:06.639970 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3613d-afc7-46d9-92b8-7c603c4dda98 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:06.818377 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087a77a-240c-4c6c-9d6b-b04a6c081c29 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:06.876796 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3613d-afc7-46d9-92b8-7c603c4dda98 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:06.891066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352065cb-be12-4e42-94a9-977b0d4f924a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:06.892299 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352065cb-be12-4e42-94a9-977b0d4f924a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:06.892964 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352065cb-be12-4e42-94a9-977b0d4f924a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:06.893610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352065cb-be12-4e42-94a9-977b0d4f924a None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:06.895246 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352065cb-be12-4e42-94a9-977b0d4f924a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:06.895807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352065cb-be12-4e42-94a9-977b0d4f924a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:06.898518 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352065cb-be12-4e42-94a9-977b0d4f924a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:06.939037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 263/530] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:06.939037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352065cb-be12-4e42-94a9-977b0d4f924a None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:06.957196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae7546-17ce-44fa-9eff-a409648aa568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:06.958305 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae7546-17ce-44fa-9eff-a409648aa568 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:06.958874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae7546-17ce-44fa-9eff-a409648aa568 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:06.959377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae7546-17ce-44fa-9eff-a409648aa568 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:07.019033 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:07.054821 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:07.092276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352065cb-be12-4e42-94a9-977b0d4f924a None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:07.102249 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae7546-17ce-44fa-9eff-a409648aa568 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:07.105293 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae7546-17ce-44fa-9eff-a409648aa568 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:07.137988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58cd8a-edeb-47f4-812e-ee69a16cf1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:07.138986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58cd8a-edeb-47f4-812e-ee69a16cf1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:07.139423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58cd8a-edeb-47f4-812e-ee69a16cf1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:07.139874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58cd8a-edeb-47f4-812e-ee69a16cf1e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:07.140891 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58cd8a-edeb-47f4-812e-ee69a16cf1e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:07.141355 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58cd8a-edeb-47f4-812e-ee69a16cf1e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:07.143073 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58cd8a-edeb-47f4-812e-ee69a16cf1e5 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:07.158866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 264/531] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:07.160157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58cd8a-edeb-47f4-812e-ee69a16cf1e5 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:07.272566 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 268/532] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:53:07.274226 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae7546-17ce-44fa-9eff-a409648aa568 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:07.292389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58cd8a-edeb-47f4-812e-ee69a16cf1e5 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:07.308836 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:07.341772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07752e0d-1a58-4b57-8f90-73cb0e46b91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:07.342634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07752e0d-1a58-4b57-8f90-73cb0e46b91f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:07.343012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07752e0d-1a58-4b57-8f90-73cb0e46b91f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:07.343347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07752e0d-1a58-4b57-8f90-73cb0e46b91f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:07.344295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07752e0d-1a58-4b57-8f90-73cb0e46b91f None admin] RBAC: Adding query filter params (role.id=29b8b950372848ffa9a75e778ee66b4c, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=ef8dcb0a0f2c4a2d955544324213739c) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:07.344686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07752e0d-1a58-4b57-8f90-73cb0e46b91f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:07.347196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07752e0d-1a58-4b57-8f90-73cb0e46b91f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:07.356006 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 265/533] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:07 2022] GET /identity/v3/role_assignments?user.id=ef8dcb0a0f2c4a2d955544324213739c&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=29b8b950372848ffa9a75e778ee66b4c => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:07.357250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07752e0d-1a58-4b57-8f90-73cb0e46b91f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:07.392796 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:07.454826 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae7546-17ce-44fa-9eff-a409648aa568 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:07.546311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ea622-7ae7-4ede-9059-97b2e6ff7896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:07.547371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ea622-7ae7-4ede-9059-97b2e6ff7896 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:07.547656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ea622-7ae7-4ede-9059-97b2e6ff7896 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:07.547948 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ea622-7ae7-4ede-9059-97b2e6ff7896 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:07.565517 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ea622-7ae7-4ede-9059-97b2e6ff7896 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:07.567860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ea622-7ae7-4ede-9059-97b2e6ff7896 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:07.589046 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07752e0d-1a58-4b57-8f90-73cb0e46b91f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:07.602827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-0e5ea622-7ae7-4ede-9059-97b2e6ff7896 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:07.602827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:07.602827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:07.602827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:07.602827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:07.602827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:07.602827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:07.602827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:07.602827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:07.602827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:07.602827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:07.602827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:07.602827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:07.602827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:07.602827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:07.604263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:07.605686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:07.605686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:07.605686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:07.605686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:07.605686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:07.605686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:07.605686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:07.605686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:07.605686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 269/534] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:07.617996 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ea622-7ae7-4ede-9059-97b2e6ff7896 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:07.692546 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3852290-a0de-40c9-b6f4-ef8a2cd60eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:07.694421 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852290-a0de-40c9-b6f4-ef8a2cd60eab None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:07.695214 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852290-a0de-40c9-b6f4-ef8a2cd60eab None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:07.696288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852290-a0de-40c9-b6f4-ef8a2cd60eab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:07.721406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3852290-a0de-40c9-b6f4-ef8a2cd60eab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:07.724851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3852290-a0de-40c9-b6f4-ef8a2cd60eab None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:07.766236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-c3852290-a0de-40c9-b6f4-ef8a2cd60eab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:07.766236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:07.766236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:07.766236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:07.766236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:07.766236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:07.766236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:07.766236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:07.766236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:07.766236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:07.766236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:07.766236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:07.766236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:07.766236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:07.766236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:07.768388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:07.770096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:07.770096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:07.770096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:07.770096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:07.774934 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 266/535] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:07.784962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3852290-a0de-40c9-b6f4-ef8a2cd60eab None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:07.882073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ea622-7ae7-4ede-9059-97b2e6ff7896 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:07.897636 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:07.962497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338e865-b0ce-4195-9e1b-ca443c68f4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:07.964750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338e865-b0ce-4195-9e1b-ca443c68f4cb None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:07.965251 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338e865-b0ce-4195-9e1b-ca443c68f4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:07.965745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338e865-b0ce-4195-9e1b-ca443c68f4cb None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:07.967492 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338e865-b0ce-4195-9e1b-ca443c68f4cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:07.968048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338e865-b0ce-4195-9e1b-ca443c68f4cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:07.979781 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338e865-b0ce-4195-9e1b-ca443c68f4cb None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:07.983354 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:08.013017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 270/536] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:08.023180 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338e865-b0ce-4195-9e1b-ca443c68f4cb None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:08.128555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3852290-a0de-40c9-b6f4-ef8a2cd60eab None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:08.173944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6d4c8-2c0c-4c57-bd8a-a833c142ef28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:08.175072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6d4c8-2c0c-4c57-bd8a-a833c142ef28 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:08.175605 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6d4c8-2c0c-4c57-bd8a-a833c142ef28 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:08.177195 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6d4c8-2c0c-4c57-bd8a-a833c142ef28 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:08.179162 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6d4c8-2c0c-4c57-bd8a-a833c142ef28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:08.179866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6d4c8-2c0c-4c57-bd8a-a833c142ef28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:08.183405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6d4c8-2c0c-4c57-bd8a-a833c142ef28 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:08.213377 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 267/537] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:08.224039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6d4c8-2c0c-4c57-bd8a-a833c142ef28 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:08.270353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338e865-b0ce-4195-9e1b-ca443c68f4cb None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:08.328376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebe60d-8c1b-479b-8d8c-aef1ad26636a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:08.329061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebe60d-8c1b-479b-8d8c-aef1ad26636a None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:08.329400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebe60d-8c1b-479b-8d8c-aef1ad26636a None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:08.329787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebe60d-8c1b-479b-8d8c-aef1ad26636a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:08.330551 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebe60d-8c1b-479b-8d8c-aef1ad26636a None admin] RBAC: Adding query filter params (role.id=29b8b950372848ffa9a75e778ee66b4c, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=9218503bc8a647d68ffab6945bae456a) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:08.330891 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebe60d-8c1b-479b-8d8c-aef1ad26636a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:08.332362 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebe60d-8c1b-479b-8d8c-aef1ad26636a None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:08.335105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:08.339553 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 271/538] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:08 2022] GET /identity/v3/role_assignments?user.id=9218503bc8a647d68ffab6945bae456a&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=29b8b950372848ffa9a75e778ee66b4c => generated 259 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:08.357073 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6d4c8-2c0c-4c57-bd8a-a833c142ef28 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:08.396612 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aba267-8430-415a-b8ef-2e943bae8ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:08.397432 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aba267-8430-415a-b8ef-2e943bae8ace None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:08.397730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aba267-8430-415a-b8ef-2e943bae8ace None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:08.398012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aba267-8430-415a-b8ef-2e943bae8ace None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:08.407066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aba267-8430-415a-b8ef-2e943bae8ace None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:08.408575 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aba267-8430-415a-b8ef-2e943bae8ace None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:08.424610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-d6aba267-8430-415a-b8ef-2e943bae8ace None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:08.424610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:08.424610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:08.424610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:08.424610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:08.424610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:08.424610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:08.424610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:08.424610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:08.424610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:08.424610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:08.424610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:08.424610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:08.424610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:08.424610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:08.426350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:08.428147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:08.428147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:08.428147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:08.428147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:08.428147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:08.428147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:08.428147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:08.428147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:08.428147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 268/539] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:08.435305 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebe60d-8c1b-479b-8d8c-aef1ad26636a None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:08.511423 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:08.554512 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebe60d-8c1b-479b-8d8c-aef1ad26636a None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:08.597934 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b947c6f-637e-4a01-b7dc-eeccc1842537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:08.598668 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b947c6f-637e-4a01-b7dc-eeccc1842537 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:08.599339 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b947c6f-637e-4a01-b7dc-eeccc1842537 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:08.599642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b947c6f-637e-4a01-b7dc-eeccc1842537 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:08.600459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b947c6f-637e-4a01-b7dc-eeccc1842537 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:08.600848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b947c6f-637e-4a01-b7dc-eeccc1842537 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:08.602434 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b947c6f-637e-4a01-b7dc-eeccc1842537 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:08.618819 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 272/540] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:08.627451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aba267-8430-415a-b8ef-2e943bae8ace None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:08.789063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aba267-8430-415a-b8ef-2e943bae8ace None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:08.847923 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16c094-d11f-4549-a718-090034252a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:08.848598 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16c094-d11f-4549-a718-090034252a65 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:08.848893 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16c094-d11f-4549-a718-090034252a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:08.849171 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16c094-d11f-4549-a718-090034252a65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:08.850012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16c094-d11f-4549-a718-090034252a65 None admin] RBAC: Adding query filter params (role.id=29b8b950372848ffa9a75e778ee66b4c, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=fd86979e0f0d4a4e9921ee5704901f74) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:08.850324 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16c094-d11f-4549-a718-090034252a65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:08.851884 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16c094-d11f-4549-a718-090034252a65 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:08.863807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 269/541] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:08 2022] GET /identity/v3/role_assignments?user.id=fd86979e0f0d4a4e9921ee5704901f74&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=29b8b950372848ffa9a75e778ee66b4c => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:09.560416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7c453-9b35-45ad-aba4-fcaa43933779 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:09.561304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7c453-9b35-45ad-aba4-fcaa43933779 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:09.561304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7c453-9b35-45ad-aba4-fcaa43933779 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:09.562849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 273/542] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:09.570184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b7b14-df0f-44de-9601-8048119a1940 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:09.570873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b7b14-df0f-44de-9601-8048119a1940 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:09.570873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b7b14-df0f-44de-9601-8048119a1940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:09.576561 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8060929-90c7-4686-b601-49fb7c33f09c None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:09.576829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8060929-90c7-4686-b601-49fb7c33f09c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:09.577124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8060929-90c7-4686-b601-49fb7c33f09c None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:09.578889 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 274/543] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:53:09.586507 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc240c-c514-482c-8ff2-87374026ab7c None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:09.587163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc240c-c514-482c-8ff2-87374026ab7c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:09.587664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc240c-c514-482c-8ff2-87374026ab7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:09.601062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-829b7b14-df0f-44de-9601-8048119a1940 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:09.602722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-829b7b14-df0f-44de-9601-8048119a1940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:09.606404 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-2cbc240c-c514-482c-8ff2-87374026ab7c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:09.608216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2cbc240c-c514-482c-8ff2-87374026ab7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:09.613286 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:09.725502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 270/544] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:09.769320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 275/545] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:09.774063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b7b14-df0f-44de-9601-8048119a1940 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:09.851874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc240c-c514-482c-8ff2-87374026ab7c None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:09.980894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b7b14-df0f-44de-9601-8048119a1940 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:09.998738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:10.059713 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c9241-abd4-4db1-83c7-111796ee4435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:10.061784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c9241-abd4-4db1-83c7-111796ee4435 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:10.063056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c9241-abd4-4db1-83c7-111796ee4435 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:10.064228 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c9241-abd4-4db1-83c7-111796ee4435 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:10.086400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:10.090683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c9241-abd4-4db1-83c7-111796ee4435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:10.108441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-843c9241-abd4-4db1-83c7-111796ee4435 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:10.108441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:10.108441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:10.108441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:10.108441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:10.108441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:10.108441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:10.108441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:10.108441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:10.108441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:10.108441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:10.108441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:10.108441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:10.108441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:10.108441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:10.108441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:10.110342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:10.110342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:10.110342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:10.110342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:10.110342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:10.110342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:10.110342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:10.110342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:10.110342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:10.110342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:10.110342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:10.110342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:10.110342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:10.110342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:10.110342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:10.110342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:10.112056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:10.112056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:10.112056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:10.112056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:10.112056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:10.112056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:10.112056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:10.112056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:10.112056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:10.112056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:10.112056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:10.112056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:10.117329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 271/546] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:10.124641 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c9241-abd4-4db1-83c7-111796ee4435 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:10.140294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc240c-c514-482c-8ff2-87374026ab7c None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:10.192251 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6ea28-0201-4af1-80eb-9c5ff24657a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:10.196049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6ea28-0201-4af1-80eb-9c5ff24657a9 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:10.196585 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6ea28-0201-4af1-80eb-9c5ff24657a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:10.197259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6ea28-0201-4af1-80eb-9c5ff24657a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:10.211078 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6ea28-0201-4af1-80eb-9c5ff24657a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:10.222163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-dfe6ea28-0201-4af1-80eb-9c5ff24657a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:10.222163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:10.222163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:10.222163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:10.222163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:10.222163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:10.222163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:10.222163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:10.222163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:10.222163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:10.222163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:10.222163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:10.222163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:10.222163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:10.222163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:10.222163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:10.223270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:10.223270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:10.223270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:10.223270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:10.223270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:10.223270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:10.223270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:10.223270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:10.223270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:10.223270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:10.223270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:10.223270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:10.223270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:10.223270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:10.223270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:10.223270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:10.224080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:10.224080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:10.224080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:10.224080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:10.224080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:10.224080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:10.224080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:10.224080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:10.224080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:10.224080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:10.224080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:10.224080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:10.228143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 276/547] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:10.239516 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6ea28-0201-4af1-80eb-9c5ff24657a9 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:10.360176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6ea28-0201-4af1-80eb-9c5ff24657a9 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:10.371895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:10.407666 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059f293-ca4a-4b6d-bb85-62d9655d2477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:10.408892 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059f293-ca4a-4b6d-bb85-62d9655d2477 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:10.409686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059f293-ca4a-4b6d-bb85-62d9655d2477 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:10.411256 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059f293-ca4a-4b6d-bb85-62d9655d2477 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:10.412507 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059f293-ca4a-4b6d-bb85-62d9655d2477 None admin] RBAC: Adding query filter params (name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:10.414816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c9241-abd4-4db1-83c7-111796ee4435 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:10.415476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059f293-ca4a-4b6d-bb85-62d9655d2477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:10.418097 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059f293-ca4a-4b6d-bb85-62d9655d2477 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:10.438803 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 277/548] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:10.448385 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059f293-ca4a-4b6d-bb85-62d9655d2477 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:10.486354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbc8f0-1bd4-44fe-ad96-573c640ec71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:10.488020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbc8f0-1bd4-44fe-ad96-573c640ec71f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:10.488702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbc8f0-1bd4-44fe-ad96-573c640ec71f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:10.489423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbc8f0-1bd4-44fe-ad96-573c640ec71f None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:10.491045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbc8f0-1bd4-44fe-ad96-573c640ec71f None admin] RBAC: Adding query filter params (name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:10.491791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbc8f0-1bd4-44fe-ad96-573c640ec71f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:10.494303 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbc8f0-1bd4-44fe-ad96-573c640ec71f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:10.510890 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 272/549] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:10.519712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbc8f0-1bd4-44fe-ad96-573c640ec71f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:10.582236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059f293-ca4a-4b6d-bb85-62d9655d2477 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:10.631003 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e930ab5-8dc9-4b02-b80b-a3749e8482d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:10.632170 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e930ab5-8dc9-4b02-b80b-a3749e8482d8 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:10.632626 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e930ab5-8dc9-4b02-b80b-a3749e8482d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:10.633040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e930ab5-8dc9-4b02-b80b-a3749e8482d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:10.659740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e930ab5-8dc9-4b02-b80b-a3749e8482d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:10.661288 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e930ab5-8dc9-4b02-b80b-a3749e8482d8 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:10.669791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:10.711909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-5e930ab5-8dc9-4b02-b80b-a3749e8482d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:10.711909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:10.711909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:10.711909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:10.711909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:10.711909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:10.711909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:10.711909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:10.711909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:10.711909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:10.711909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:10.711909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:10.711909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:10.711909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:10.711909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:10.713008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:10.714466 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:10.714466 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:10.714466 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:10.714466 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:10.715057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbc8f0-1bd4-44fe-ad96-573c640ec71f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:10.715149 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 278/550] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:10.720980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e930ab5-8dc9-4b02-b80b-a3749e8482d8 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:10.779423 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:10.794386 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddbc1f-69c1-4596-b914-4c069b5958ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:10.795225 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddbc1f-69c1-4596-b914-4c069b5958ff None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:10.795505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddbc1f-69c1-4596-b914-4c069b5958ff None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:10.795780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddbc1f-69c1-4596-b914-4c069b5958ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:10.820003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddbc1f-69c1-4596-b914-4c069b5958ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:10.821529 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddbc1f-69c1-4596-b914-4c069b5958ff None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:10.857344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-01ddbc1f-69c1-4596-b914-4c069b5958ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:10.857344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:10.857344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:10.857344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:10.857344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:10.857344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:10.857344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:10.857344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:10.857344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:10.857344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:10.857344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:10.857344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:10.857344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:10.857344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:10.857344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:10.858608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:10.859488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:10.859488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:10.859488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:10.859488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:10.859488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 273/551] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:10.871770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddbc1f-69c1-4596-b914-4c069b5958ff None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:10.920568 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e930ab5-8dc9-4b02-b80b-a3749e8482d8 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:10.962350 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff409d8-b053-488f-842c-78f9767695a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:10.963673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff409d8-b053-488f-842c-78f9767695a1 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:10.964034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff409d8-b053-488f-842c-78f9767695a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:10.964407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff409d8-b053-488f-842c-78f9767695a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:10.965428 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff409d8-b053-488f-842c-78f9767695a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:10.965813 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff409d8-b053-488f-842c-78f9767695a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:10.967344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff409d8-b053-488f-842c-78f9767695a1 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:10.983563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 279/552] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:10.983563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e437fb-ee45-4afe-8533-7e45a02d9bc2 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:10.983563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e437fb-ee45-4afe-8533-7e45a02d9bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:10.983563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e437fb-ee45-4afe-8533-7e45a02d9bc2 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:10.987186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 280/553] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:10.988298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e437fb-ee45-4afe-8533-7e45a02d9bc2 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:11.025099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddbc1f-69c1-4596-b914-4c069b5958ff None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:11.042917 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:11.066240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24db163-34ff-40d5-8662-5b6bc8848fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:11.067002 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24db163-34ff-40d5-8662-5b6bc8848fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:11.067297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24db163-34ff-40d5-8662-5b6bc8848fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:11.067585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24db163-34ff-40d5-8662-5b6bc8848fd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:11.068328 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24db163-34ff-40d5-8662-5b6bc8848fd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:11.068638 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24db163-34ff-40d5-8662-5b6bc8848fd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:11.070136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24db163-34ff-40d5-8662-5b6bc8848fd7 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:11.084242 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 274/554] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:11.085621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e01e5-48ac-4d66-b157-338bad53229a None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:11.091280 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e01e5-48ac-4d66-b157-338bad53229a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:11.091647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e01e5-48ac-4d66-b157-338bad53229a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:11.111465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-d35e01e5-48ac-4d66-b157-338bad53229a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:11.112752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d35e01e5-48ac-4d66-b157-338bad53229a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:11.118592 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:11.151206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e437fb-ee45-4afe-8533-7e45a02d9bc2 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:11.202283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 275/555] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:11.203510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e01e5-48ac-4d66-b157-338bad53229a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:11.226810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7b1af-ff54-45bb-8bf1-2b6ff0619070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:11.227661 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7b1af-ff54-45bb-8bf1-2b6ff0619070 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:11.228038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7b1af-ff54-45bb-8bf1-2b6ff0619070 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:11.228328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7b1af-ff54-45bb-8bf1-2b6ff0619070 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:11.247832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7b1af-ff54-45bb-8bf1-2b6ff0619070 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:11.250386 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7b1af-ff54-45bb-8bf1-2b6ff0619070 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:11.274204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-79b7b1af-ff54-45bb-8bf1-2b6ff0619070 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:53:11.274204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:11.274204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:11.274204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:11.274204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:11.274204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:11.274204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:11.274204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:11.274204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:11.274204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:11.274204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:11.274204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:11.274204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:11.274204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:11.274204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:11.274204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:11.277082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:11.279473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:11.279473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:11.279473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:11.279473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:11.279473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:11.279473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:11.279473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:11.279473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:11.279473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:53:11.279473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:11.280824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 281/556] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:53:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:11.282249 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7b1af-ff54-45bb-8bf1-2b6ff0619070 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:11.354559 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e01e5-48ac-4d66-b157-338bad53229a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:11.357055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:11.392702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca82547-f421-41c5-aded-8c688cccb3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:11.393508 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82547-f421-41c5-aded-8c688cccb3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:11.393620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7b1af-ff54-45bb-8bf1-2b6ff0619070 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:11.393804 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82547-f421-41c5-aded-8c688cccb3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:11.394073 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82547-f421-41c5-aded-8c688cccb3f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:11.402922 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca82547-f421-41c5-aded-8c688cccb3f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:11.404406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca82547-f421-41c5-aded-8c688cccb3f4 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:11.420354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-cca82547-f421-41c5-aded-8c688cccb3f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:53:11.420354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:11.420354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:11.420354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:11.420354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:11.420354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:11.420354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:11.420354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:11.420354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:11.420354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:11.420354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:11.420354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:11.420354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:11.420354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:11.420354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:11.420354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:11.421388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:11.422824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:11.422824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:11.422824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:11.422824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:11.422824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:11.422824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:11.422824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:11.422824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:11.422824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:53:11.422824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:11.422824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 276/557] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Jul 18 14:53:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:11.423419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633cb71-5dae-47b7-8857-5e77810b026a None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:11.424015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633cb71-5dae-47b7-8857-5e77810b026a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:11.424015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633cb71-5dae-47b7-8857-5e77810b026a None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:11.429177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 277/558] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:11.429177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633cb71-5dae-47b7-8857-5e77810b026a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:11.441548 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e325b-3e75-4916-bb30-cf2f9c2669d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:11.442743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e325b-3e75-4916-bb30-cf2f9c2669d0 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:11.443219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e325b-3e75-4916-bb30-cf2f9c2669d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:11.443726 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e325b-3e75-4916-bb30-cf2f9c2669d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:11.458249 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e325b-3e75-4916-bb30-cf2f9c2669d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:11.466610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-5b9e325b-3e75-4916-bb30-cf2f9c2669d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:11.466610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:11.466610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:11.466610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:11.466610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:11.466610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:11.466610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:11.466610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:11.466610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:11.466610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:11.466610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:11.466610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:11.466610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:11.466610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:11.466610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:11.466610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:11.467700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:11.467700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:11.467700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:11.467700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:11.467700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:11.467700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:11.467700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:11.467700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:11.467700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:11.467700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:11.467700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:11.467700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:11.467700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:11.467700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:11.467700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:11.467700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:11.468631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:11.468631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:11.468631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:11.468631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:11.468631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:11.468631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:11.468631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:11.468631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:11.468631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:11.468631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:11.468631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:11.468631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:11.468631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 282/559] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:11.469364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e325b-3e75-4916-bb30-cf2f9c2669d0 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:11.472468 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:11.567447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633cb71-5dae-47b7-8857-5e77810b026a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:11.594050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e325b-3e75-4916-bb30-cf2f9c2669d0 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:11.608649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d3d39-5ed9-480a-841c-9fb941c51771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:11.609318 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d3d39-5ed9-480a-841c-9fb941c51771 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:11.609601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d3d39-5ed9-480a-841c-9fb941c51771 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:11.609873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d3d39-5ed9-480a-841c-9fb941c51771 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:11.610745 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d3d39-5ed9-480a-841c-9fb941c51771 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:11.611051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d3d39-5ed9-480a-841c-9fb941c51771 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:11.612514 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d3d39-5ed9-480a-841c-9fb941c51771 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:11.631082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a6924-aec2-491e-8b66-f2e61a2d287d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:11.636095 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 278/560] 10.222.0.83 () {56 vars in 1135 bytes} [Mon Jul 18 14:53:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:11.636272 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a6924-aec2-491e-8b66-f2e61a2d287d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:11.636373 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f4ee0-7689-4c68-abf5-94fab1b9bcc3 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:11.636462 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a6924-aec2-491e-8b66-f2e61a2d287d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:11.636548 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f4ee0-7689-4c68-abf5-94fab1b9bcc3 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:11.636632 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a6924-aec2-491e-8b66-f2e61a2d287d None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:11.636716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f4ee0-7689-4c68-abf5-94fab1b9bcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:11.636813 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a6924-aec2-491e-8b66-f2e61a2d287d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:11.636813 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a6924-aec2-491e-8b66-f2e61a2d287d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:11.636972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a6924-aec2-491e-8b66-f2e61a2d287d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:11.653098 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-7f9f4ee0-7689-4c68-abf5-94fab1b9bcc3 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:11.654400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7f9f4ee0-7689-4c68-abf5-94fab1b9bcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:11.656623 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 283/561] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Jul 18 14:53:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:11.657247 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a6924-aec2-491e-8b66-f2e61a2d287d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:11.678689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:11.720547 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:11.731569 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 279/562] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:11.732463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f4ee0-7689-4c68-abf5-94fab1b9bcc3 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:11.762172 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a6924-aec2-491e-8b66-f2e61a2d287d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:11.798873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63835f4e-258e-487d-b4cf-11ad7845d6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:11.799552 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63835f4e-258e-487d-b4cf-11ad7845d6cc None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:11.799843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63835f4e-258e-487d-b4cf-11ad7845d6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:11.800126 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63835f4e-258e-487d-b4cf-11ad7845d6cc None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:11.800886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63835f4e-258e-487d-b4cf-11ad7845d6cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:11.801199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63835f4e-258e-487d-b4cf-11ad7845d6cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:11.802621 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63835f4e-258e-487d-b4cf-11ad7845d6cc None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:11.810927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 284/563] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:11.812453 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63835f4e-258e-487d-b4cf-11ad7845d6cc None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:11.857403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f4ee0-7689-4c68-abf5-94fab1b9bcc3 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:11.895109 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31213c4e-8a68-4896-8906-41f133a1a66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:11.895772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31213c4e-8a68-4896-8906-41f133a1a66c None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:11.896060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31213c4e-8a68-4896-8906-41f133a1a66c None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:11.896340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31213c4e-8a68-4896-8906-41f133a1a66c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:11.919213 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:11.924970 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63835f4e-258e-487d-b4cf-11ad7845d6cc None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:11.947615 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:11.960121 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74551334-47d6-4db2-a33a-2fb9175aeaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:11.960987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74551334-47d6-4db2-a33a-2fb9175aeaa7 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:11.961274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74551334-47d6-4db2-a33a-2fb9175aeaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:11.961544 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74551334-47d6-4db2-a33a-2fb9175aeaa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:11.968009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31213c4e-8a68-4896-8906-41f133a1a66c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:11.969869 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31213c4e-8a68-4896-8906-41f133a1a66c None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:11.973345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74551334-47d6-4db2-a33a-2fb9175aeaa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:11.974774 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74551334-47d6-4db2-a33a-2fb9175aeaa7 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:11.992469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-74551334-47d6-4db2-a33a-2fb9175aeaa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:11.992469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:11.992469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:11.992469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:11.992469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:11.992469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:11.992469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:11.992469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:11.992469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:11.992469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:11.992469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:11.992469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:11.992469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:11.992469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:11.992469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:11.993590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:11.995320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:11.995320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:11.995320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:11.995320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:11.995320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 285/564] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:11.996016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a8da3-03fc-4dd4-a3f2-87628665c510 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:11.996016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a8da3-03fc-4dd4-a3f2-87628665c510 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:11.996307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a8da3-03fc-4dd4-a3f2-87628665c510 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:12.001248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 286/565] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:12.002267 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a8da3-03fc-4dd4-a3f2-87628665c510 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:12.085496 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 280/566] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:53:12.086346 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31213c4e-8a68-4896-8906-41f133a1a66c None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:12.118594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a8da3-03fc-4dd4-a3f2-87628665c510 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:12.145292 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:12.156415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36ff54-2054-42bd-8846-2c2a6ca0c902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:12.157344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36ff54-2054-42bd-8846-2c2a6ca0c902 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:12.157634 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36ff54-2054-42bd-8846-2c2a6ca0c902 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:12.157915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36ff54-2054-42bd-8846-2c2a6ca0c902 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:12.171711 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36ff54-2054-42bd-8846-2c2a6ca0c902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:12.180210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-1b36ff54-2054-42bd-8846-2c2a6ca0c902 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:12.180210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:12.180210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:12.180210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:12.180210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:12.180210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:12.180210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:12.180210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:12.180210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:12.180210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:12.180210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:12.180210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:12.180210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:12.180210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:12.180210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:12.180210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:12.181340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:12.181340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:12.181340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:12.181340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:12.181340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:12.181340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:12.181340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:12.181340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:12.181340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:12.181340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:12.181340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:12.181340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:12.181340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:12.181340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:12.181340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:12.181340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:12.182298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:12.182298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:12.182298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:12.182298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:12.182298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:12.182298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:12.182298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:12.182298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:12.182298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:12.182298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:12.182298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:12.182298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:12.182298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 287/567] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:12.183081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36ff54-2054-42bd-8846-2c2a6ca0c902 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:12.205723 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31213c4e-8a68-4896-8906-41f133a1a66c None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:12.221339 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:12.245203 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3fc81-d5b9-4e19-94ef-8323ff65d9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:12.246072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3fc81-d5b9-4e19-94ef-8323ff65d9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:12.246356 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3fc81-d5b9-4e19-94ef-8323ff65d9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:12.246694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3fc81-d5b9-4e19-94ef-8323ff65d9c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:12.271861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3fc81-d5b9-4e19-94ef-8323ff65d9c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:12.274406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3fc81-d5b9-4e19-94ef-8323ff65d9c1 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:12.293928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-30a3fc81-d5b9-4e19-94ef-8323ff65d9c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:12.293928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:12.293928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:12.293928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:12.293928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:12.293928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:12.293928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:12.293928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:12.293928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:12.293928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:12.293928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:12.293928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:12.293928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:12.293928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:12.293928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:12.295233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:12.296159 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:12.296159 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:12.296159 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:12.296159 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:12.296159 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 281/568] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:12.296517 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8672fd5-c9b6-4285-af74-e025716ba55d None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:12.296717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8672fd5-c9b6-4285-af74-e025716ba55d None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:12.296932 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8672fd5-c9b6-4285-af74-e025716ba55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:12.315389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-a8672fd5-c9b6-4285-af74-e025716ba55d None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:12.316625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a8672fd5-c9b6-4285-af74-e025716ba55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:12.317864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36ff54-2054-42bd-8846-2c2a6ca0c902 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:12.356405 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f413a59-d30a-4c37-8f37-dfdf02e1a1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:12.357072 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f413a59-d30a-4c37-8f37-dfdf02e1a1ca None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:12.357356 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f413a59-d30a-4c37-8f37-dfdf02e1a1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:12.357636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f413a59-d30a-4c37-8f37-dfdf02e1a1ca None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:12.358649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f413a59-d30a-4c37-8f37-dfdf02e1a1ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:12.358962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f413a59-d30a-4c37-8f37-dfdf02e1a1ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:12.360360 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f413a59-d30a-4c37-8f37-dfdf02e1a1ca None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:12.378453 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 288/569] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:12.379132 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f413a59-d30a-4c37-8f37-dfdf02e1a1ca None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:12.394019 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 282/570] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:12.394992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8672fd5-c9b6-4285-af74-e025716ba55d None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:12.430570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:12.492175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:12.501706 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f413a59-d30a-4c37-8f37-dfdf02e1a1ca None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:12.519872 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8672fd5-c9b6-4285-af74-e025716ba55d None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:12.543820 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ac0f7-d97b-432a-9909-16221247f82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:12.544633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ac0f7-d97b-432a-9909-16221247f82c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:12.544914 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ac0f7-d97b-432a-9909-16221247f82c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:12.545187 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ac0f7-d97b-432a-9909-16221247f82c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:12.556896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ac0f7-d97b-432a-9909-16221247f82c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:12.559321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ac0f7-d97b-432a-9909-16221247f82c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:12.565697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc57ec2-8d6d-4fb5-a169-1a69e1c98ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:12.566405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc57ec2-8d6d-4fb5-a169-1a69e1c98ebd None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:12.566737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc57ec2-8d6d-4fb5-a169-1a69e1c98ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:12.567023 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc57ec2-8d6d-4fb5-a169-1a69e1c98ebd None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:12.567811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc57ec2-8d6d-4fb5-a169-1a69e1c98ebd None admin] RBAC: Adding query filter params (name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:12.568144 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc57ec2-8d6d-4fb5-a169-1a69e1c98ebd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:12.569563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc57ec2-8d6d-4fb5-a169-1a69e1c98ebd None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:12.579944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 283/571] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:12.580918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc57ec2-8d6d-4fb5-a169-1a69e1c98ebd None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:12.581274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-ca6ac0f7-d97b-432a-9909-16221247f82c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:12.581274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:12.581274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:12.581274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:12.581274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:12.581274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:12.581274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:12.581274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:12.581274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:12.581274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:12.581274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:12.581274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:12.581274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:12.581274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:12.581274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:12.582366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:12.583336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:12.583336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:12.583336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:12.583336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:12.583336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 289/572] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:12.584213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ac0f7-d97b-432a-9909-16221247f82c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:12.696790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:12.703364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc57ec2-8d6d-4fb5-a169-1a69e1c98ebd None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:12.703592 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ac0f7-d97b-432a-9909-16221247f82c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:12.745048 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c164d2-beab-4fa5-b87f-147e1e50c36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:12.745656 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c164d2-beab-4fa5-b87f-147e1e50c36c None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:12.745942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c164d2-beab-4fa5-b87f-147e1e50c36c None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:12.746228 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c164d2-beab-4fa5-b87f-147e1e50c36c None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:12.747000 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c164d2-beab-4fa5-b87f-147e1e50c36c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:12.747310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c164d2-beab-4fa5-b87f-147e1e50c36c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:12.748398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc2cda-91e1-4692-ad5d-9f05277a8b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:12.748716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c164d2-beab-4fa5-b87f-147e1e50c36c None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:12.749367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc2cda-91e1-4692-ad5d-9f05277a8b89 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:12.749720 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc2cda-91e1-4692-ad5d-9f05277a8b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:12.750061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc2cda-91e1-4692-ad5d-9f05277a8b89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:12.758103 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc2cda-91e1-4692-ad5d-9f05277a8b89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:12.759763 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc2cda-91e1-4692-ad5d-9f05277a8b89 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:12.761377 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 284/573] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:12.762238 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c164d2-beab-4fa5-b87f-147e1e50c36c None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:12.774981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-84bc2cda-91e1-4692-ad5d-9f05277a8b89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:12.774981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:12.774981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:12.774981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:12.774981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:12.774981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:12.774981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:12.774981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:12.774981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:12.774981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:12.774981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:12.774981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:12.774981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:12.774981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:12.774981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:12.776369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:12.782431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:12.782431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:12.782431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:12.782431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:12.782431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:12.782431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:12.782431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:12.782431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:12.782431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 290/574] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:12.782431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc2cda-91e1-4692-ad5d-9f05277a8b89 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:12.794239 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:12.882372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c164d2-beab-4fa5-b87f-147e1e50c36c None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:12.894224 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc2cda-91e1-4692-ad5d-9f05277a8b89 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:12.918333 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9a3b4-3aca-4d6a-afad-117b47cbd4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:12.919188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9a3b4-3aca-4d6a-afad-117b47cbd4da None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:12.919471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9a3b4-3aca-4d6a-afad-117b47cbd4da None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:12.919752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9a3b4-3aca-4d6a-afad-117b47cbd4da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:12.929997 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c59985-9fdc-41d2-87f1-7cdb3f8aac02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:12.930842 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c59985-9fdc-41d2-87f1-7cdb3f8aac02 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:12.931131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c59985-9fdc-41d2-87f1-7cdb3f8aac02 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:12.931552 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c59985-9fdc-41d2-87f1-7cdb3f8aac02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:12.932858 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c9a3b4-3aca-4d6a-afad-117b47cbd4da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:12.941902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-97c9a3b4-3aca-4d6a-afad-117b47cbd4da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:12.941902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:12.941902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:12.941902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:12.941902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:12.941902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:12.941902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:12.941902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:12.941902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:12.941902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:12.941902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:12.941902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:12.941902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:12.941902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:12.941902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:12.941902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:12.942993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:12.942993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:12.942993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:12.942993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:12.942993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:12.942993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:12.942993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:12.942993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:12.942993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:12.942993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:12.942993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:12.942993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:12.942993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:12.942993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:12.942993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:12.942993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:12.943851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:12.943851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:12.943851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:12.943851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:12.943851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:12.943851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:12.943851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:12.943851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:12.943851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:12.943851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:12.943851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:12.943851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:12.943851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 285/575] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:12.944578 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9a3b4-3aca-4d6a-afad-117b47cbd4da None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:12.945857 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c59985-9fdc-41d2-87f1-7cdb3f8aac02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:12.947447 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c59985-9fdc-41d2-87f1-7cdb3f8aac02 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:12.964288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:12.967070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-b4c59985-9fdc-41d2-87f1-7cdb3f8aac02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:12.967070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:12.967070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:12.967070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:12.967070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:12.967070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:12.967070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:12.967070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:12.967070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:12.967070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:12.967070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:12.967070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:12.967070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:12.967070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:12.967070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:12.968160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:12.969108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:12.969108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:12.969108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:12.969108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:12.969108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 291/576] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:12.969409 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c59985-9fdc-41d2-87f1-7cdb3f8aac02 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:13.036720 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:13.053498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9a3b4-3aca-4d6a-afad-117b47cbd4da None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:13.080454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c59985-9fdc-41d2-87f1-7cdb3f8aac02 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:13.092324 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55a5ff-97a3-4111-ba22-b89fa9af7c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:13.093021 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55a5ff-97a3-4111-ba22-b89fa9af7c6e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:13.093279 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55a5ff-97a3-4111-ba22-b89fa9af7c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:13.093557 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55a5ff-97a3-4111-ba22-b89fa9af7c6e None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:13.094289 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55a5ff-97a3-4111-ba22-b89fa9af7c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:13.094628 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55a5ff-97a3-4111-ba22-b89fa9af7c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:13.096048 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55a5ff-97a3-4111-ba22-b89fa9af7c6e None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:13.110038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 286/577] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:13.111336 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55a5ff-97a3-4111-ba22-b89fa9af7c6e None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:13.117343 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e659a54-82af-4b9b-a2f2-cb8c786f6011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:13.118148 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659a54-82af-4b9b-a2f2-cb8c786f6011 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:13.118438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659a54-82af-4b9b-a2f2-cb8c786f6011 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:13.118752 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659a54-82af-4b9b-a2f2-cb8c786f6011 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:13.127157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e659a54-82af-4b9b-a2f2-cb8c786f6011 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:13.128644 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e659a54-82af-4b9b-a2f2-cb8c786f6011 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:13.144921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-2e659a54-82af-4b9b-a2f2-cb8c786f6011 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 14:53:13.144921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:13.144921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:13.144921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:13.144921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:13.144921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:13.144921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:13.144921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:13.144921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:13.144921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:13.144921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:13.144921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:13.144921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:13.144921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:13.144921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:13.144921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:13.146017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:13.147073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:13.147073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:13.147073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:13.147073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:13.147073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:13.147073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:13.147073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:13.147073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:13.147073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 14:53:13.147073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:13.147073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 292/578] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:53:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:13.147073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e659a54-82af-4b9b-a2f2-cb8c786f6011 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:13.209784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:13.235092 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55a5ff-97a3-4111-ba22-b89fa9af7c6e None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:13.256444 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e659a54-82af-4b9b-a2f2-cb8c786f6011 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:13.270896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67433a-b2f5-45d6-94a5-5cebfb2f15ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:13.271557 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67433a-b2f5-45d6-94a5-5cebfb2f15ed None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:13.271875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67433a-b2f5-45d6-94a5-5cebfb2f15ed None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:13.272119 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67433a-b2f5-45d6-94a5-5cebfb2f15ed None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:13.272903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67433a-b2f5-45d6-94a5-5cebfb2f15ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:13.273211 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67433a-b2f5-45d6-94a5-5cebfb2f15ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:13.274720 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67433a-b2f5-45d6-94a5-5cebfb2f15ed None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:13.276515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:13.291881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 287/579] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:13.291881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67433a-b2f5-45d6-94a5-5cebfb2f15ed None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:13.296822 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e9bdf-bac0-4d8d-9ff8-e412b9876320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:13.297550 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e9bdf-bac0-4d8d-9ff8-e412b9876320 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:13.297825 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e9bdf-bac0-4d8d-9ff8-e412b9876320 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:13.298103 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e9bdf-bac0-4d8d-9ff8-e412b9876320 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:13.298925 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e9bdf-bac0-4d8d-9ff8-e412b9876320 None admin] RBAC: Adding query filter params (name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:13.299238 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e9bdf-bac0-4d8d-9ff8-e412b9876320 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:13.300687 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e9bdf-bac0-4d8d-9ff8-e412b9876320 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:13.310642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 293/580] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:13.311550 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e9bdf-bac0-4d8d-9ff8-e412b9876320 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:13.408666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67433a-b2f5-45d6-94a5-5cebfb2f15ed None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:13.425369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e9bdf-bac0-4d8d-9ff8-e412b9876320 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:13.446013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090e7b0-320f-4668-b8a0-13238261ed2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:13.446714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090e7b0-320f-4668-b8a0-13238261ed2c None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:13.447824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090e7b0-320f-4668-b8a0-13238261ed2c None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:13.448117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090e7b0-320f-4668-b8a0-13238261ed2c None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:13.449016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090e7b0-320f-4668-b8a0-13238261ed2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:13.449324 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090e7b0-320f-4668-b8a0-13238261ed2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:13.450764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090e7b0-320f-4668-b8a0-13238261ed2c None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:13.453255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:13.465914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 288/581] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:13.466826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090e7b0-320f-4668-b8a0-13238261ed2c None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:13.468521 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7af78e-7acf-46fa-9e5f-9bb743576fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:13.469310 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7af78e-7acf-46fa-9e5f-9bb743576fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:13.469648 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7af78e-7acf-46fa-9e5f-9bb743576fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:13.469937 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7af78e-7acf-46fa-9e5f-9bb743576fc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:13.477515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7af78e-7acf-46fa-9e5f-9bb743576fc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:13.479158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7af78e-7acf-46fa-9e5f-9bb743576fc5 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:13.494073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-4c7af78e-7acf-46fa-9e5f-9bb743576fc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:13.494073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:13.494073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:13.494073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:13.494073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:13.494073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:13.494073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:13.494073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:13.494073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:13.494073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:13.494073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:13.494073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:13.494073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:13.494073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:13.494073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:13.496270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:13.497571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:13.497571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:13.497571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:13.497571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:13.497571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:13.497571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:13.497571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:13.497571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:13.497571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 294/582] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:13.497571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7af78e-7acf-46fa-9e5f-9bb743576fc5 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:13.543208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:13.582784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090e7b0-320f-4668-b8a0-13238261ed2c None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:13.606611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7af78e-7acf-46fa-9e5f-9bb743576fc5 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:13.622463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3f199-5247-4cdb-9830-33deaa4b0ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:13.623194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3f199-5247-4cdb-9830-33deaa4b0ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:13.623481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3f199-5247-4cdb-9830-33deaa4b0ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:13.623755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3f199-5247-4cdb-9830-33deaa4b0ec4 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:13.624606 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3f199-5247-4cdb-9830-33deaa4b0ec4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:13.624914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3f199-5247-4cdb-9830-33deaa4b0ec4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:13.626407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3f199-5247-4cdb-9830-33deaa4b0ec4 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:13.643997 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e049071-deca-41fb-84b1-f8c4af6c45c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:13.644830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 289/583] 10.222.0.83 () {56 vars in 1137 bytes} [Mon Jul 18 14:53:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:13.644986 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e049071-deca-41fb-84b1-f8c4af6c45c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:13.645168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e049071-deca-41fb-84b1-f8c4af6c45c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:13.645446 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e049071-deca-41fb-84b1-f8c4af6c45c9 None admin] PATH_INFO: `/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/7e1355d18fb049468651b3675c217d60/roles/29b8b950372848ffa9a75e778ee66b4c` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:13.645652 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3f199-5247-4cdb-9830-33deaa4b0ec4 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:13.662367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e049071-deca-41fb-84b1-f8c4af6c45c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f777cd6a0c744e594f8e29ab58cf8bd, user_id=7e1355d18fb049468651b3675c217d60, role_id=29b8b950372848ffa9a75e778ee66b4c)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:13.664043 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e049071-deca-41fb-84b1-f8c4af6c45c9 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:13.702693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:13.740070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 295/584] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:53:13 2022] PUT /identity/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/7e1355d18fb049468651b3675c217d60/roles/29b8b950372848ffa9a75e778ee66b4c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:53:13.740913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e049071-deca-41fb-84b1-f8c4af6c45c9 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:13.748584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3f199-5247-4cdb-9830-33deaa4b0ec4 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:13.791085 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20ac54-61f0-41ee-a67d-a689acd66630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:13.791991 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20ac54-61f0-41ee-a67d-a689acd66630 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:13.792324 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20ac54-61f0-41ee-a67d-a689acd66630 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:13.792606 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20ac54-61f0-41ee-a67d-a689acd66630 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:13.807439 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20ac54-61f0-41ee-a67d-a689acd66630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:13.808942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20ac54-61f0-41ee-a67d-a689acd66630 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:13.830770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-3c20ac54-61f0-41ee-a67d-a689acd66630 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:13.830770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:13.830770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:13.830770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:13.830770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:13.830770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:13.830770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:13.830770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:13.830770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:13.830770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:13.830770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:13.830770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:13.830770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:13.830770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:13.830770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:13.832102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:13.833212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:13.833212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:13.833212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:13.833212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:13.833212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 290/585] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:13.833212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20ac54-61f0-41ee-a67d-a689acd66630 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:13.881325 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:13.895450 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e049071-deca-41fb-84b1-f8c4af6c45c9 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:13.944886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419bd1f-f37b-4c1f-8a5d-29ae5df88723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:13.946157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419bd1f-f37b-4c1f-8a5d-29ae5df88723 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:13.946430 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419bd1f-f37b-4c1f-8a5d-29ae5df88723 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:13.946734 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419bd1f-f37b-4c1f-8a5d-29ae5df88723 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:13.955437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e419bd1f-f37b-4c1f-8a5d-29ae5df88723 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:13.956910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e419bd1f-f37b-4c1f-8a5d-29ae5df88723 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:13.959624 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20ac54-61f0-41ee-a67d-a689acd66630 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:13.967627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:13.973872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-e419bd1f-f37b-4c1f-8a5d-29ae5df88723 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 14:53:13.973872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:13.973872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:13.973872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:13.973872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:13.973872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:13.973872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:13.973872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:13.973872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:13.973872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:13.973872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:13.973872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:13.973872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:13.973872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:13.973872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:13.973872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:13.975741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:13.976724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:13.976724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:13.976724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:13.976724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:13.976724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:13.976724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:13.976724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:13.976724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:13.976724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 14:53:13.976724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:13.976724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 296/586] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:53:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:13.976724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419bd1f-f37b-4c1f-8a5d-29ae5df88723 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:14.001770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782766cc-e6c1-4d85-b7cb-83bbce71bd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:14.002444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782766cc-e6c1-4d85-b7cb-83bbce71bd95 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:14.002763 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782766cc-e6c1-4d85-b7cb-83bbce71bd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:14.003046 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782766cc-e6c1-4d85-b7cb-83bbce71bd95 None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:14.003882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782766cc-e6c1-4d85-b7cb-83bbce71bd95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:14.004579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782766cc-e6c1-4d85-b7cb-83bbce71bd95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:14.006434 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782766cc-e6c1-4d85-b7cb-83bbce71bd95 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:14.024793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 291/587] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:14.024793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782766cc-e6c1-4d85-b7cb-83bbce71bd95 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:14.117671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419bd1f-f37b-4c1f-8a5d-29ae5df88723 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:14.151506 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782766cc-e6c1-4d85-b7cb-83bbce71bd95 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:14.155640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76020d54-a143-42fa-81d3-470aa34a6843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:14.156454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76020d54-a143-42fa-81d3-470aa34a6843 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:14.156743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76020d54-a143-42fa-81d3-470aa34a6843 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:14.157023 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76020d54-a143-42fa-81d3-470aa34a6843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:14.170012 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76020d54-a143-42fa-81d3-470aa34a6843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:14.171516 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76020d54-a143-42fa-81d3-470aa34a6843 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:14.188777 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb37afa-4b02-472b-960f-c0c80b62fd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:14.189450 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb37afa-4b02-472b-960f-c0c80b62fd6a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:14.189734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb37afa-4b02-472b-960f-c0c80b62fd6a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:14.190008 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb37afa-4b02-472b-960f-c0c80b62fd6a None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:14.190764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb37afa-4b02-472b-960f-c0c80b62fd6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:14.191074 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb37afa-4b02-472b-960f-c0c80b62fd6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:14.191376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-76020d54-a143-42fa-81d3-470aa34a6843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:14.191376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:14.191376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:14.191376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:14.191376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:14.191376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:14.191376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:14.191376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:14.191376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:14.191376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:14.191376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:14.191376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:14.191376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:14.191376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:14.191376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:14.192438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:14.199013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:14.199013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:14.199013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:14.199013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:14.199252 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb37afa-4b02-472b-960f-c0c80b62fd6a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:14.199331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 297/588] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:14.199331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76020d54-a143-42fa-81d3-470aa34a6843 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:14.199331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:14.215370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 292/589] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:14.216313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb37afa-4b02-472b-960f-c0c80b62fd6a None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:14.258527 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:14.327145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76020d54-a143-42fa-81d3-470aa34a6843 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:14.347784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb37afa-4b02-472b-960f-c0c80b62fd6a None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:14.368650 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7a19b-4729-4793-99e4-3683c6e76886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:14.370006 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7a19b-4729-4793-99e4-3683c6e76886 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:14.370300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7a19b-4729-4793-99e4-3683c6e76886 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:14.370605 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7a19b-4729-4793-99e4-3683c6e76886 None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:14.371450 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7a19b-4729-4793-99e4-3683c6e76886 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:14.371765 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7a19b-4729-4793-99e4-3683c6e76886 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:14.373232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7a19b-4729-4793-99e4-3683c6e76886 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:14.387032 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3a49d-daca-497e-a371-241d520a0362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:14.387720 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3a49d-daca-497e-a371-241d520a0362 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:14.387987 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3a49d-daca-497e-a371-241d520a0362 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:14.388316 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3a49d-daca-497e-a371-241d520a0362 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:14.389132 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3a49d-daca-497e-a371-241d520a0362 None admin] RBAC: Adding query filter params (role.id=29b8b950372848ffa9a75e778ee66b4c, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=f88dfe620ff04ffe88088a8a23c7d6e9) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:14.389542 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3a49d-daca-497e-a371-241d520a0362 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:14.391603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3a49d-daca-497e-a371-241d520a0362 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:14.402737 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 298/590] 10.222.0.83 () {56 vars in 1137 bytes} [Mon Jul 18 14:53:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:14.402737 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7a19b-4729-4793-99e4-3683c6e76886 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:14.403019 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 293/591] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:14 2022] GET /identity/v3/role_assignments?user.id=f88dfe620ff04ffe88088a8a23c7d6e9&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=29b8b950372848ffa9a75e778ee66b4c => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:14.407616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3a49d-daca-497e-a371-241d520a0362 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:14.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:14.540769 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:14.555254 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7a19b-4729-4793-99e4-3683c6e76886 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:14.555992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3a49d-daca-497e-a371-241d520a0362 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:14.600950 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24be5f-ec9c-46c2-98f0-978606da88fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:14.601214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6506b8-940b-4218-a44d-b470871d0484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:14.601655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6506b8-940b-4218-a44d-b470871d0484 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:14.601945 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6506b8-940b-4218-a44d-b470871d0484 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:14.602229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6506b8-940b-4218-a44d-b470871d0484 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:14.603015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24be5f-ec9c-46c2-98f0-978606da88fd None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:14.603203 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6506b8-940b-4218-a44d-b470871d0484 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:14.603904 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6506b8-940b-4218-a44d-b470871d0484 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:14.604387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24be5f-ec9c-46c2-98f0-978606da88fd None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:14.605073 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24be5f-ec9c-46c2-98f0-978606da88fd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:14.605613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6506b8-940b-4218-a44d-b470871d0484 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:14.613803 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24be5f-ec9c-46c2-98f0-978606da88fd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:14.615656 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24be5f-ec9c-46c2-98f0-978606da88fd None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:14.624315 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 299/592] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:14.624315 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6506b8-940b-4218-a44d-b470871d0484 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:14.637115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-7f24be5f-ec9c-46c2-98f0-978606da88fd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 14:53:14.637115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:14.637115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:14.637115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:14.637115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:14.637115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:14.637115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:14.637115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:14.637115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:14.637115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:14.637115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:14.637115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:14.637115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:14.637115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:14.637115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:14.637115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:14.638135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:14.639136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:14.639136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:14.639136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:14.639136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:14.639136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:14.639136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:14.639136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:14.639136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:14.639136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 14:53:14.639136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:14.641807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 294/593] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:14.642901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24be5f-ec9c-46c2-98f0-978606da88fd None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:14.755255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6506b8-940b-4218-a44d-b470871d0484 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:14.768294 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24be5f-ec9c-46c2-98f0-978606da88fd None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:14.783817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:14.802552 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb93fb-178e-4007-8e6d-6934ec47ab4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:14.803706 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb93fb-178e-4007-8e6d-6934ec47ab4f None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:14.803996 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb93fb-178e-4007-8e6d-6934ec47ab4f None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:14.804277 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb93fb-178e-4007-8e6d-6934ec47ab4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:14.812282 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996b644-640b-4f1d-9771-587d228ea5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:14.813429 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996b644-640b-4f1d-9771-587d228ea5da None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:14.813935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996b644-640b-4f1d-9771-587d228ea5da None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:14.814447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996b644-640b-4f1d-9771-587d228ea5da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:14.818078 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb93fb-178e-4007-8e6d-6934ec47ab4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:14.819920 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb93fb-178e-4007-8e6d-6934ec47ab4f None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:14.824420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996b644-640b-4f1d-9771-587d228ea5da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:14.826280 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996b644-640b-4f1d-9771-587d228ea5da None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:14.839872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-1bcb93fb-178e-4007-8e6d-6934ec47ab4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:14.839872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:14.839872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:14.839872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:14.839872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:14.839872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:14.839872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:14.839872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:14.839872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:14.839872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:14.839872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:14.839872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:14.839872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:14.839872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:14.839872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:14.840883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:14.841805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:14.841805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:14.841805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:14.841805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:14.845314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-4996b644-640b-4f1d-9771-587d228ea5da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:14.845314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:14.845314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:14.845314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:14.845314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:14.845314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:14.845314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:14.845314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:14.845314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:14.845314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:14.845314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:14.845314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:14.845314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:14.845314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:14.845314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:14.846813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:14.847747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:14.847747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:14.847747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:14.847747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:14.847747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:14.847747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:14.847747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:14.847747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:14.848187 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 300/594] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:14.848187 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb93fb-178e-4007-8e6d-6934ec47ab4f None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:14.850128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 295/595] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:14.851484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996b644-640b-4f1d-9771-587d228ea5da None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:14.876815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:14.983801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb93fb-178e-4007-8e6d-6934ec47ab4f None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:15.004882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996b644-640b-4f1d-9771-587d228ea5da None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:15.025044 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625082b-4ccd-43d2-996c-4ea6c6f3fd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:15.026209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625082b-4ccd-43d2-996c-4ea6c6f3fd49 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:15.026778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625082b-4ccd-43d2-996c-4ea6c6f3fd49 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:15.027331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625082b-4ccd-43d2-996c-4ea6c6f3fd49 None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:15.028505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625082b-4ccd-43d2-996c-4ea6c6f3fd49 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:15.029037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625082b-4ccd-43d2-996c-4ea6c6f3fd49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:15.030843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625082b-4ccd-43d2-996c-4ea6c6f3fd49 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:15.049524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837b4b7-f0f5-4f50-b51e-82cccb293186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:15.050208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837b4b7-f0f5-4f50-b51e-82cccb293186 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:15.050498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837b4b7-f0f5-4f50-b51e-82cccb293186 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:15.050813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837b4b7-f0f5-4f50-b51e-82cccb293186 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:15.051612 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3837b4b7-f0f5-4f50-b51e-82cccb293186 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:15.051926 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3837b4b7-f0f5-4f50-b51e-82cccb293186 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:15.052866 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 301/596] 10.222.0.83 () {56 vars in 1139 bytes} [Mon Jul 18 14:53:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:15.053335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3837b4b7-f0f5-4f50-b51e-82cccb293186 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:15.054364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625082b-4ccd-43d2-996c-4ea6c6f3fd49 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:15.070635 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 296/597] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:15.072517 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837b4b7-f0f5-4f50-b51e-82cccb293186 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:15.099323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:15.174279 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:15.188197 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625082b-4ccd-43d2-996c-4ea6c6f3fd49 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:15.215932 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837b4b7-f0f5-4f50-b51e-82cccb293186 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:15.239687 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f39be-c307-4ec2-9ad4-3836323b26f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:15.240367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f39be-c307-4ec2-9ad4-3836323b26f3 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:15.240656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f39be-c307-4ec2-9ad4-3836323b26f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:15.240937 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f39be-c307-4ec2-9ad4-3836323b26f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:15.241741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f39be-c307-4ec2-9ad4-3836323b26f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:15.242054 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f39be-c307-4ec2-9ad4-3836323b26f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:15.244655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f39be-c307-4ec2-9ad4-3836323b26f3 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:15.259504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53feeb2-8db7-408c-b591-22d59b930a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:15.260700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53feeb2-8db7-408c-b591-22d59b930a76 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:15.261208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53feeb2-8db7-408c-b591-22d59b930a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:15.261814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53feeb2-8db7-408c-b591-22d59b930a76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:15.265769 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 302/598] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:15.266699 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f39be-c307-4ec2-9ad4-3836323b26f3 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:15.276712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53feeb2-8db7-408c-b591-22d59b930a76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:15.278604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53feeb2-8db7-408c-b591-22d59b930a76 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:15.299212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-b53feeb2-8db7-408c-b591-22d59b930a76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:15.299212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:15.299212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:15.299212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:15.299212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:15.299212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:15.299212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:15.299212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:15.299212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:15.299212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:15.299212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:15.299212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:15.299212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:15.299212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:15.299212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:15.302272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:15.303237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:15.303237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:15.303237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:15.303237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:15.304104 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 297/599] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:15.305603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53feeb2-8db7-408c-b591-22d59b930a76 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:15.388946 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:15.401143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f39be-c307-4ec2-9ad4-3836323b26f3 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:15.439055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53feeb2-8db7-408c-b591-22d59b930a76 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:15.446069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5dc73-9ae3-4d84-98af-7afe3b0c5391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:15.447826 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5dc73-9ae3-4d84-98af-7afe3b0c5391 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:15.448465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5dc73-9ae3-4d84-98af-7afe3b0c5391 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:15.448993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5dc73-9ae3-4d84-98af-7afe3b0c5391 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:15.457307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5dc73-9ae3-4d84-98af-7afe3b0c5391 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:15.459185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5dc73-9ae3-4d84-98af-7afe3b0c5391 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:15.468706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:15.477062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-cdc5dc73-9ae3-4d84-98af-7afe3b0c5391 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:15.477062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:15.477062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:15.477062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:15.477062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:15.477062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:15.477062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:15.477062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:15.477062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:15.477062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:15.477062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:15.477062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:15.477062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:15.477062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:15.477062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:15.478076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:15.479816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:15.479816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:15.479816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:15.479816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:15.479816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:15.479816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:15.479816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:15.479816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:15.483161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 303/600] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:15.484322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5dc73-9ae3-4d84-98af-7afe3b0c5391 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:15.499811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee7439-c33b-4bed-95e3-459d90fb4377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:15.501541 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee7439-c33b-4bed-95e3-459d90fb4377 None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:15.501831 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee7439-c33b-4bed-95e3-459d90fb4377 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:15.502116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee7439-c33b-4bed-95e3-459d90fb4377 None admin] PATH_INFO: `/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/ef8dcb0a0f2c4a2d955544324213739c/roles/29b8b950372848ffa9a75e778ee66b4c` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:15.524868 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee7439-c33b-4bed-95e3-459d90fb4377 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f777cd6a0c744e594f8e29ab58cf8bd, user_id=ef8dcb0a0f2c4a2d955544324213739c, role_id=29b8b950372848ffa9a75e778ee66b4c)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:15.527775 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee7439-c33b-4bed-95e3-459d90fb4377 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:15.604746 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 298/601] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:53:15 2022] PUT /identity/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/ef8dcb0a0f2c4a2d955544324213739c/roles/29b8b950372848ffa9a75e778ee66b4c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:53:15.605962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee7439-c33b-4bed-95e3-459d90fb4377 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:15.640438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5dc73-9ae3-4d84-98af-7afe3b0c5391 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:15.735347 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac564ea5-e7f7-48bc-932a-c036858a987b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:15.736043 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac564ea5-e7f7-48bc-932a-c036858a987b None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:15.736329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac564ea5-e7f7-48bc-932a-c036858a987b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:15.736612 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac564ea5-e7f7-48bc-932a-c036858a987b None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:15.737359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac564ea5-e7f7-48bc-932a-c036858a987b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:15.737666 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac564ea5-e7f7-48bc-932a-c036858a987b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:15.739115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac564ea5-e7f7-48bc-932a-c036858a987b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:15.771272 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 304/602] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:15.779705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac564ea5-e7f7-48bc-932a-c036858a987b None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:15.794080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee7439-c33b-4bed-95e3-459d90fb4377 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:15.811060 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:15.837017 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a1ebd-d094-4b47-b4b0-e3e719c98491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:15.838363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a1ebd-d094-4b47-b4b0-e3e719c98491 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:15.838969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a1ebd-d094-4b47-b4b0-e3e719c98491 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:15.839495 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a1ebd-d094-4b47-b4b0-e3e719c98491 None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:15.840566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a1ebd-d094-4b47-b4b0-e3e719c98491 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:15.841114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a1ebd-d094-4b47-b4b0-e3e719c98491 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:15.842978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a1ebd-d094-4b47-b4b0-e3e719c98491 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:15.858375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 299/603] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:15.867085 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a1ebd-d094-4b47-b4b0-e3e719c98491 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:15.942026 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:15.983618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac564ea5-e7f7-48bc-932a-c036858a987b None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:16.044009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa69e71-75a1-4d06-81df-a17d37c8f67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:16.044831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa69e71-75a1-4d06-81df-a17d37c8f67b None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:16.045156 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa69e71-75a1-4d06-81df-a17d37c8f67b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:16.045528 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa69e71-75a1-4d06-81df-a17d37c8f67b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:16.055870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa69e71-75a1-4d06-81df-a17d37c8f67b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:16.058417 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa69e71-75a1-4d06-81df-a17d37c8f67b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:16.065713 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a1ebd-d094-4b47-b4b0-e3e719c98491 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:16.077814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-8aa69e71-75a1-4d06-81df-a17d37c8f67b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:16.077814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:16.077814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:16.077814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:16.077814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:16.077814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:16.077814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:16.077814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:16.077814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:16.077814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:16.077814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:16.077814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:16.077814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:16.077814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:16.077814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:16.080056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:16.081463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:16.081463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:16.081463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:16.081463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:16.081463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:16.081463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:16.081463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:16.081463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:16.081463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 305/604] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:16.081463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c1684-25f5-4b42-8b41-40965fcb1ec7 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:16.081463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c1684-25f5-4b42-8b41-40965fcb1ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:16.082693 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c1684-25f5-4b42-8b41-40965fcb1ec7 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:16.083478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 306/605] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:16.088527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c1684-25f5-4b42-8b41-40965fcb1ec7 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:16.123456 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1598cce-3d1c-4c86-a5ae-9cb2a98586a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:16.124393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1598cce-3d1c-4c86-a5ae-9cb2a98586a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:16.124944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1598cce-3d1c-4c86-a5ae-9cb2a98586a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:16.125223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1598cce-3d1c-4c86-a5ae-9cb2a98586a2 None admin] PATH_INFO: `/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/9218503bc8a647d68ffab6945bae456a/roles/29b8b950372848ffa9a75e778ee66b4c` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:16.148841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1598cce-3d1c-4c86-a5ae-9cb2a98586a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f777cd6a0c744e594f8e29ab58cf8bd, user_id=9218503bc8a647d68ffab6945bae456a, role_id=29b8b950372848ffa9a75e778ee66b4c)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:16.151577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1598cce-3d1c-4c86-a5ae-9cb2a98586a2 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:16.188965 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:16.224193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c1684-25f5-4b42-8b41-40965fcb1ec7 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:16.266645 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4b89c-16c6-4307-8abc-244ea1443bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:16.267321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4b89c-16c6-4307-8abc-244ea1443bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:16.267604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4b89c-16c6-4307-8abc-244ea1443bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:16.267884 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4b89c-16c6-4307-8abc-244ea1443bb1 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:16.268897 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4b89c-16c6-4307-8abc-244ea1443bb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:16.270358 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4b89c-16c6-4307-8abc-244ea1443bb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:16.271856 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4b89c-16c6-4307-8abc-244ea1443bb1 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:16.287336 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 300/606] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:53:15 2022] PUT /identity/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/9218503bc8a647d68ffab6945bae456a/roles/29b8b950372848ffa9a75e778ee66b4c => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:53:16.289246 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748d7c0-bad4-4cd4-a714-2457788ec483 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:16.289533 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748d7c0-bad4-4cd4-a714-2457788ec483 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:16.291287 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748d7c0-bad4-4cd4-a714-2457788ec483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:16.304946 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 307/607] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:16.311069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4b89c-16c6-4307-8abc-244ea1443bb1 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:16.336300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-f748d7c0-bad4-4cd4-a714-2457788ec483 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:16.339363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f748d7c0-bad4-4cd4-a714-2457788ec483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:16.465294 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 301/608] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:16.499718 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748d7c0-bad4-4cd4-a714-2457788ec483 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:16.542267 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4b89c-16c6-4307-8abc-244ea1443bb1 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:16.545269 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:16.585629 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:16.592287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda3db8-981b-4f1f-adf1-bf75a1f14de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:16.593137 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda3db8-981b-4f1f-adf1-bf75a1f14de3 None admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:16.593532 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda3db8-981b-4f1f-adf1-bf75a1f14de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:16.593722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda3db8-981b-4f1f-adf1-bf75a1f14de3 None admin] PATH_INFO: `/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/fd86979e0f0d4a4e9921ee5704901f74/roles/29b8b950372848ffa9a75e778ee66b4c` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:16.617764 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda3db8-981b-4f1f-adf1-bf75a1f14de3 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f777cd6a0c744e594f8e29ab58cf8bd, user_id=fd86979e0f0d4a4e9921ee5704901f74, role_id=29b8b950372848ffa9a75e778ee66b4c)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:16.620581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda3db8-981b-4f1f-adf1-bf75a1f14de3 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:16.657841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748d7c0-bad4-4cd4-a714-2457788ec483 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:16.714064 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 308/609] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:53:16 2022] PUT /identity/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/fd86979e0f0d4a4e9921ee5704901f74/roles/29b8b950372848ffa9a75e778ee66b4c => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:53:16.718859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df35acf-60ba-4553-a0c2-e981ed58d6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:16.719861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df35acf-60ba-4553-a0c2-e981ed58d6ce None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:16.720251 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df35acf-60ba-4553-a0c2-e981ed58d6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:16.720653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df35acf-60ba-4553-a0c2-e981ed58d6ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:16.738231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df35acf-60ba-4553-a0c2-e981ed58d6ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:16.748224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-1df35acf-60ba-4553-a0c2-e981ed58d6ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:16.748224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:16.748224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:16.748224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:16.748224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:16.748224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:16.748224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:16.748224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:16.748224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:16.748224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:16.748224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:16.748224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:16.748224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:16.748224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:16.748224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:16.748224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:16.749455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:16.749455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:16.749455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:16.749455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:16.749455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:16.749455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:16.749455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:16.749455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:16.749455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:16.749455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:16.749455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:16.749455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:16.749455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:16.749455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:16.749455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:16.749455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:16.750467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:16.750467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:16.750467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:16.750467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:16.750467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:16.750467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:16.750467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:16.750467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:16.750467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:16.750467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:16.750467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:16.750467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:16.754573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 302/610] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:16.761481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda3db8-981b-4f1f-adf1-bf75a1f14de3 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:16.787692 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868cb60-723a-4623-a7e4-ae374638c605 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:16.788152 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868cb60-723a-4623-a7e4-ae374638c605 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:16.788490 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868cb60-723a-4623-a7e4-ae374638c605 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:16.790180 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 303/611] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:16.796931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfaf5fa-e23b-4a5a-8329-e7c90b822057 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:16.797584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfaf5fa-e23b-4a5a-8329-e7c90b822057 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:16.797584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfaf5fa-e23b-4a5a-8329-e7c90b822057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:16.831458 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-1cfaf5fa-e23b-4a5a-8329-e7c90b822057 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:16.832761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1cfaf5fa-e23b-4a5a-8329-e7c90b822057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:16.975846 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 304/612] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:53:16.984836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda3db8-981b-4f1f-adf1-bf75a1f14de3 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:17.011543 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfaf5fa-e23b-4a5a-8329-e7c90b822057 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:17.014040 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:17.053903 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb8c41-d6a4-47bb-b57e-9b85e6fdf365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:17.054912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb8c41-d6a4-47bb-b57e-9b85e6fdf365 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:17.055394 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb8c41-d6a4-47bb-b57e-9b85e6fdf365 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:17.055833 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb8c41-d6a4-47bb-b57e-9b85e6fdf365 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:17.066885 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb8c41-d6a4-47bb-b57e-9b85e6fdf365 None admin] RBAC: Adding query filter params (name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:17.067457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb8c41-d6a4-47bb-b57e-9b85e6fdf365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:17.069125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb8c41-d6a4-47bb-b57e-9b85e6fdf365 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:17.084294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 309/613] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:17.090586 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb8c41-d6a4-47bb-b57e-9b85e6fdf365 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:17.170936 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:17.245408 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfaf5fa-e23b-4a5a-8329-e7c90b822057 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:17.297262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb8c41-d6a4-47bb-b57e-9b85e6fdf365 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:17.331272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bef40-73ed-4ee7-bc31-b70b05c125e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:17.332418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bef40-73ed-4ee7-bc31-b70b05c125e3 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:17.332868 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bef40-73ed-4ee7-bc31-b70b05c125e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:17.333305 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bef40-73ed-4ee7-bc31-b70b05c125e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:17.340156 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349f15a-e5c6-45fe-9aa2-cccba517b74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:17.341090 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349f15a-e5c6-45fe-9aa2-cccba517b74c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:17.341461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349f15a-e5c6-45fe-9aa2-cccba517b74c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:17.341808 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349f15a-e5c6-45fe-9aa2-cccba517b74c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:17.363031 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6349f15a-e5c6-45fe-9aa2-cccba517b74c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:17.364108 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bef40-73ed-4ee7-bc31-b70b05c125e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:17.367146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6349f15a-e5c6-45fe-9aa2-cccba517b74c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:17.395372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-cc7bef40-73ed-4ee7-bc31-b70b05c125e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:17.395372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:17.395372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:17.395372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:17.395372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:17.395372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:17.395372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:17.395372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:17.395372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:17.395372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:17.395372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:17.395372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:17.395372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:17.395372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:17.395372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:17.395372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:17.396654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:17.396654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:17.396654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:17.396654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:17.396654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:17.396654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:17.396654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:17.396654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:17.396654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:17.396654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:17.396654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:17.396654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:17.396654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:17.396654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:17.396654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:17.396654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:17.403188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:17.403188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:17.403188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:17.403188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:17.403188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:17.403188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:17.403188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:17.403188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:17.403188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:17.403188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:17.403188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:17.403188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:17.403188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 305/614] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:17.403188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bef40-73ed-4ee7-bc31-b70b05c125e3 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:17.418989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-6349f15a-e5c6-45fe-9aa2-cccba517b74c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:17.418989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:17.418989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:17.418989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:17.418989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:17.418989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:17.418989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:17.418989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:17.418989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:17.418989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:17.418989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:17.418989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:17.418989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:17.418989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:17.418989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:17.420169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:17.421344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:17.421344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:17.421344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:17.421344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:17.431053 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 310/615] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:17.436880 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349f15a-e5c6-45fe-9aa2-cccba517b74c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:17.476001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:17.523729 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bef40-73ed-4ee7-bc31-b70b05c125e3 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:17.604320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e77dbe-d1d6-4263-a444-122b8ad57791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:17.605016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e77dbe-d1d6-4263-a444-122b8ad57791 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:17.605418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e77dbe-d1d6-4263-a444-122b8ad57791 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:17.605704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e77dbe-d1d6-4263-a444-122b8ad57791 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:17.606471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e77dbe-d1d6-4263-a444-122b8ad57791 None admin] RBAC: Adding query filter params (name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:17.606807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e77dbe-d1d6-4263-a444-122b8ad57791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:17.608504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e77dbe-d1d6-4263-a444-122b8ad57791 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:17.624958 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 306/616] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:17.630719 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e77dbe-d1d6-4263-a444-122b8ad57791 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:17.660679 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:17.692596 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349f15a-e5c6-45fe-9aa2-cccba517b74c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:17.752891 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e77dbe-d1d6-4263-a444-122b8ad57791 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:17.780720 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:17.800406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d0e1f-4707-4e99-85ae-7aedfe3e3ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:17.801272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d0e1f-4707-4e99-85ae-7aedfe3e3ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:17.801513 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d0e1f-4707-4e99-85ae-7aedfe3e3ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:17.801795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d0e1f-4707-4e99-85ae-7aedfe3e3ab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:17.803046 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c37e7-0985-4789-9df0-9fddc4c2ccfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:17.803728 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c37e7-0985-4789-9df0-9fddc4c2ccfb None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:17.804012 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c37e7-0985-4789-9df0-9fddc4c2ccfb None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:17.804289 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c37e7-0985-4789-9df0-9fddc4c2ccfb None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:17.805040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c37e7-0985-4789-9df0-9fddc4c2ccfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:17.805353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c37e7-0985-4789-9df0-9fddc4c2ccfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:17.807492 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c37e7-0985-4789-9df0-9fddc4c2ccfb None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:17.828956 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d0e1f-4707-4e99-85ae-7aedfe3e3ab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:17.829837 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 311/617] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:17.830470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d0e1f-4707-4e99-85ae-7aedfe3e3ab4 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:17.839803 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c37e7-0985-4789-9df0-9fddc4c2ccfb None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:17.872625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-a26d0e1f-4707-4e99-85ae-7aedfe3e3ab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:17.872625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:17.872625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:17.872625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:17.872625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:17.872625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:17.872625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:17.872625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:17.872625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:17.872625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:17.872625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:17.872625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:17.872625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:17.872625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:17.872625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:17.873801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:17.874734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:17.874734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:17.874734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:17.874734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:17.874734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 307/618] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:17.882599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d0e1f-4707-4e99-85ae-7aedfe3e3ab4 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:17.961276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c37e7-0985-4789-9df0-9fddc4c2ccfb None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:18.002218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec57511-9fa8-45c8-b619-da9d91d9266a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:18.004046 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec57511-9fa8-45c8-b619-da9d91d9266a None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:18.006587 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec57511-9fa8-45c8-b619-da9d91d9266a None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:18.006883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec57511-9fa8-45c8-b619-da9d91d9266a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:18.023027 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec57511-9fa8-45c8-b619-da9d91d9266a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:18.024529 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec57511-9fa8-45c8-b619-da9d91d9266a None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:18.046199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-4ec57511-9fa8-45c8-b619-da9d91d9266a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:53:18.046199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:18.046199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:18.046199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:18.046199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:18.046199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:18.046199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:18.046199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:18.046199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:18.046199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:18.046199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:18.046199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:18.046199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:18.046199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:18.046199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:18.046199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:18.047229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:18.048207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:18.048207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:18.048207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:18.048207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:18.048207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:18.048207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:18.048207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:18.048207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:18.048207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:53:18.048207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:18.048207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 312/619] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Jul 18 14:53:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:18.055840 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec57511-9fa8-45c8-b619-da9d91d9266a None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:18.091372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:18.154996 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d0e1f-4707-4e99-85ae-7aedfe3e3ab4 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:18.235064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3b439-6af7-45a2-b5f7-001d8d6ee307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:18.236157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3b439-6af7-45a2-b5f7-001d8d6ee307 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:18.236658 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3b439-6af7-45a2-b5f7-001d8d6ee307 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:18.237139 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3b439-6af7-45a2-b5f7-001d8d6ee307 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:18.238197 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3b439-6af7-45a2-b5f7-001d8d6ee307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:18.238761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3b439-6af7-45a2-b5f7-001d8d6ee307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:18.240402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3b439-6af7-45a2-b5f7-001d8d6ee307 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:18.251084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec57511-9fa8-45c8-b619-da9d91d9266a None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:18.256763 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 308/620] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:18.263708 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3b439-6af7-45a2-b5f7-001d8d6ee307 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:18.324192 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba9c66-7ede-4fa9-8a07-a1bb1882bb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:18.324778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba9c66-7ede-4fa9-8a07-a1bb1882bb9e None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:18.325071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba9c66-7ede-4fa9-8a07-a1bb1882bb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:18.325354 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba9c66-7ede-4fa9-8a07-a1bb1882bb9e None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:18.326219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba9c66-7ede-4fa9-8a07-a1bb1882bb9e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:18.326548 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba9c66-7ede-4fa9-8a07-a1bb1882bb9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:18.328020 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba9c66-7ede-4fa9-8a07-a1bb1882bb9e None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:18.337375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:18.360294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 313/621] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Jul 18 14:53:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:18.371575 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba9c66-7ede-4fa9-8a07-a1bb1882bb9e None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:18.416574 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3b439-6af7-45a2-b5f7-001d8d6ee307 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:18.453937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53e891-5b75-49d1-925c-98367a4eba0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:18.454841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53e891-5b75-49d1-925c-98367a4eba0d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:18.455130 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53e891-5b75-49d1-925c-98367a4eba0d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:18.455412 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53e891-5b75-49d1-925c-98367a4eba0d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:18.458568 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:18.464173 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53e891-5b75-49d1-925c-98367a4eba0d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:18.465687 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53e891-5b75-49d1-925c-98367a4eba0d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:18.481313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-2a53e891-5b75-49d1-925c-98367a4eba0d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:53:18.481313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:18.481313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:18.481313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:18.481313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:18.481313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:18.481313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:18.481313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:18.481313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:18.481313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:18.481313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:18.481313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:18.481313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:18.481313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:18.481313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:18.481313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:18.482515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:18.483963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:18.483963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:18.483963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:18.483963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:18.483963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:18.483963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:18.483963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:18.483963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:18.483963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:53:18.483963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:18.483963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 309/622] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:53:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:18.484689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3a426-e349-437d-8879-9f28ad4c0785 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:18.488306 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3a426-e349-437d-8879-9f28ad4c0785 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:18.488306 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3a426-e349-437d-8879-9f28ad4c0785 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:18.488306 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 310/623] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:18.489129 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3a426-e349-437d-8879-9f28ad4c0785 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:18.500801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba9c66-7ede-4fa9-8a07-a1bb1882bb9e None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:18.543690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b1672-04b2-407a-b5f9-2f0e7d67876b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:18.544459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b1672-04b2-407a-b5f9-2f0e7d67876b None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:18.544751 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b1672-04b2-407a-b5f9-2f0e7d67876b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:18.545035 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b1672-04b2-407a-b5f9-2f0e7d67876b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:18.597389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:18.604646 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3a426-e349-437d-8879-9f28ad4c0785 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:18.626609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b1672-04b2-407a-b5f9-2f0e7d67876b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:18.628376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b1672-04b2-407a-b5f9-2f0e7d67876b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:18.642045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb98f4b-a674-4a59-b7d2-dea472e8520e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:18.642748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb98f4b-a674-4a59-b7d2-dea472e8520e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:18.643097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb98f4b-a674-4a59-b7d2-dea472e8520e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:18.643381 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb98f4b-a674-4a59-b7d2-dea472e8520e None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:18.644237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb98f4b-a674-4a59-b7d2-dea472e8520e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:18.644546 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb98f4b-a674-4a59-b7d2-dea472e8520e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:18.646107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb98f4b-a674-4a59-b7d2-dea472e8520e None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:18.670698 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 311/624] 10.222.0.83 () {56 vars in 1135 bytes} [Mon Jul 18 14:53:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:18.670698 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78297337-77e2-41e9-a0df-afc321206aa4 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:18.670698 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78297337-77e2-41e9-a0df-afc321206aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:18.670698 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78297337-77e2-41e9-a0df-afc321206aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:18.695870 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-78297337-77e2-41e9-a0df-afc321206aa4 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:18.697772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-78297337-77e2-41e9-a0df-afc321206aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:18.717847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:18.754239 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 314/625] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:53:18.755470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b1672-04b2-407a-b5f9-2f0e7d67876b None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:18.776181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 312/626] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:18.776181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78297337-77e2-41e9-a0df-afc321206aa4 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:18.871267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:18.888271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78297337-77e2-41e9-a0df-afc321206aa4 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:18.893152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b1672-04b2-407a-b5f9-2f0e7d67876b None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:18.925145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfff93-e212-40e3-924c-26c9e9d477d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:18.925957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfff93-e212-40e3-924c-26c9e9d477d9 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:18.926245 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfff93-e212-40e3-924c-26c9e9d477d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:18.926530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfff93-e212-40e3-924c-26c9e9d477d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:18.934593 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590dba0a-7aa7-4109-8891-dbe2cfef5661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:18.935402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dba0a-7aa7-4109-8891-dbe2cfef5661 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:18.935686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dba0a-7aa7-4109-8891-dbe2cfef5661 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:18.935963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dba0a-7aa7-4109-8891-dbe2cfef5661 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:18.939029 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfff93-e212-40e3-924c-26c9e9d477d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:18.940681 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfff93-e212-40e3-924c-26c9e9d477d9 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:18.948679 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590dba0a-7aa7-4109-8891-dbe2cfef5661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:18.949679 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590dba0a-7aa7-4109-8891-dbe2cfef5661 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:18.960754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-7ecfff93-e212-40e3-924c-26c9e9d477d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:18.960754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:18.960754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:18.960754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:18.960754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:18.960754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:18.960754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:18.960754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:18.960754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:18.960754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:18.960754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:18.960754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:18.960754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:18.960754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:18.960754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:18.961886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:18.963681 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:18.963681 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:18.963681 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:18.963681 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:18.966707 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 313/627] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:18.966707 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfff93-e212-40e3-924c-26c9e9d477d9 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:18.968344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-590dba0a-7aa7-4109-8891-dbe2cfef5661 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:18.968344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:18.968344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:18.968344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:18.968344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:18.968344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:18.968344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:18.968344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:18.968344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:18.968344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:18.968344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:18.968344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:18.968344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:18.968344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:18.968344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:18.969242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:18.970227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:18.970227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:18.970227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:18.970227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:18.970227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 315/628] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:18.971161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0436c-c6ac-4644-bf2b-5a4ff63911d9 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:18.971450 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0436c-c6ac-4644-bf2b-5a4ff63911d9 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:18.971735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0436c-c6ac-4644-bf2b-5a4ff63911d9 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:18.973132 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 316/629] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:18.973684 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0436c-c6ac-4644-bf2b-5a4ff63911d9 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:19.002121 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:19.084778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfff93-e212-40e3-924c-26c9e9d477d9 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:19.085138 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0436c-c6ac-4644-bf2b-5a4ff63911d9 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:19.124030 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28160c19-28d9-43fc-b868-a12a91a8565d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:19.124611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28160c19-28d9-43fc-b868-a12a91a8565d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:19.124714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea88b61-063f-4c05-827a-add9a55aa9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:19.124902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28160c19-28d9-43fc-b868-a12a91a8565d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:19.125187 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28160c19-28d9-43fc-b868-a12a91a8565d None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:19.125475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea88b61-063f-4c05-827a-add9a55aa9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:19.125762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea88b61-063f-4c05-827a-add9a55aa9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:19.125924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28160c19-28d9-43fc-b868-a12a91a8565d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:19.126042 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea88b61-063f-4c05-827a-add9a55aa9d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:19.126236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28160c19-28d9-43fc-b868-a12a91a8565d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:19.127740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28160c19-28d9-43fc-b868-a12a91a8565d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:19.129363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:19.140112 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea88b61-063f-4c05-827a-add9a55aa9d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:19.140319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 317/630] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:19.140997 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28160c19-28d9-43fc-b868-a12a91a8565d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:19.148827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-5ea88b61-063f-4c05-827a-add9a55aa9d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:19.148827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:19.148827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:19.148827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:19.148827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:19.148827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:19.148827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:19.148827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:19.148827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:19.148827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:19.148827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:19.148827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:19.148827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:19.148827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:19.148827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:19.148827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:19.151084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:19.151084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:19.151084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:19.151084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:19.151084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:19.151084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:19.151084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:19.151084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:19.151084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:19.151084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:19.151084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:19.151084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:19.151084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:19.151084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:19.151084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:19.151084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:19.152387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:19.152387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:19.152387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:19.152387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:19.152387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:19.152387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:19.152387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:19.152387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:19.152387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:19.152387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:19.152387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:19.152387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:19.152387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 314/631] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:19.152387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d018e-e5b7-41b2-b33b-1ce96ce1158f None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:19.153353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d018e-e5b7-41b2-b33b-1ce96ce1158f None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:19.153353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d018e-e5b7-41b2-b33b-1ce96ce1158f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:19.173756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-1a3d018e-e5b7-41b2-b33b-1ce96ce1158f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:19.175574 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1a3d018e-e5b7-41b2-b33b-1ce96ce1158f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:19.243687 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:19.260151 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 315/632] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:19.261488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28160c19-28d9-43fc-b868-a12a91a8565d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:19.261763 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb6a24-a4c5-47c7-9155-cb0fec2ad530 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:19.262131 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb6a24-a4c5-47c7-9155-cb0fec2ad530 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:19.262513 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb6a24-a4c5-47c7-9155-cb0fec2ad530 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:19.264049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 316/633] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:19.264710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb6a24-a4c5-47c7-9155-cb0fec2ad530 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:19.305600 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451d5e5-eeba-48ca-8305-cb3bccf496d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:19.306640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451d5e5-eeba-48ca-8305-cb3bccf496d1 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:19.306977 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451d5e5-eeba-48ca-8305-cb3bccf496d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:19.307259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451d5e5-eeba-48ca-8305-cb3bccf496d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:19.308028 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451d5e5-eeba-48ca-8305-cb3bccf496d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:19.308338 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451d5e5-eeba-48ca-8305-cb3bccf496d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:19.309822 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451d5e5-eeba-48ca-8305-cb3bccf496d1 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:19.322587 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 318/634] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:19.323518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451d5e5-eeba-48ca-8305-cb3bccf496d1 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:19.406401 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb6a24-a4c5-47c7-9155-cb0fec2ad530 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:19.435920 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:19.444657 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451d5e5-eeba-48ca-8305-cb3bccf496d1 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:19.445016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048e451-2161-42d2-89b0-c4bdd1efee5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:19.445931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048e451-2161-42d2-89b0-c4bdd1efee5b None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:19.446272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048e451-2161-42d2-89b0-c4bdd1efee5b None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:19.447259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048e451-2161-42d2-89b0-c4bdd1efee5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:19.454602 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2048e451-2161-42d2-89b0-c4bdd1efee5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:19.456182 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2048e451-2161-42d2-89b0-c4bdd1efee5b None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:19.471020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-2048e451-2161-42d2-89b0-c4bdd1efee5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:19.471020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:19.471020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:19.471020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:19.471020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:19.471020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:19.471020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:19.471020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:19.471020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:19.471020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:19.471020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:19.471020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:19.471020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:19.471020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:19.471020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:19.472094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:19.473003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:19.473003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:19.473003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:19.473003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:19.473003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:19.473003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:19.473003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:19.473003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:19.474868 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 317/635] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:19.476132 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96529ed5-7604-44df-b407-4b09ce0dc6e8 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:19.476424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96529ed5-7604-44df-b407-4b09ce0dc6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:19.476823 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96529ed5-7604-44df-b407-4b09ce0dc6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:19.482180 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5bd624-be69-42e4-922e-b7f930bc9bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:19.483180 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bd624-be69-42e4-922e-b7f930bc9bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:19.483470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bd624-be69-42e4-922e-b7f930bc9bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:19.483751 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bd624-be69-42e4-922e-b7f930bc9bf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:19.484499 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5bd624-be69-42e4-922e-b7f930bc9bf6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:19.484809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5bd624-be69-42e4-922e-b7f930bc9bf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:19.486183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5bd624-be69-42e4-922e-b7f930bc9bf6 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:19.494309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 319/636] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:19.494457 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-96529ed5-7604-44df-b407-4b09ce0dc6e8 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:19.495153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5bd624-be69-42e4-922e-b7f930bc9bf6 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:19.495714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-96529ed5-7604-44df-b407-4b09ce0dc6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:19.504950 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:19.564488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 318/637] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:19.565335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96529ed5-7604-44df-b407-4b09ce0dc6e8 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:19.600358 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5bd624-be69-42e4-922e-b7f930bc9bf6 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:19.639059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ab245-1162-47fe-a1a5-a96536f19121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:19.639914 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ab245-1162-47fe-a1a5-a96536f19121 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:19.640198 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ab245-1162-47fe-a1a5-a96536f19121 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:19.640473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ab245-1162-47fe-a1a5-a96536f19121 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:19.651661 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ab245-1162-47fe-a1a5-a96536f19121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:19.659578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-246ab245-1162-47fe-a1a5-a96536f19121 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:19.659578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:19.659578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:19.659578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:19.659578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:19.659578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:19.659578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:19.659578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:19.659578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:19.659578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:19.659578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:19.659578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:19.659578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:19.659578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:19.659578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:19.659578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:19.660755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:19.660755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:19.660755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:19.660755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:19.660755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:19.660755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:19.660755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:19.660755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:19.660755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:19.660755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:19.660755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:19.660755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:19.660755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:19.660755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:19.660755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:19.660755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:19.661640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:19.661640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:19.661640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:19.661640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:19.661640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:19.661640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:19.661640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:19.661640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:19.661640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:19.661640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:19.661640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:19.661640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:19.661640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 320/638] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:19.662374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ab245-1162-47fe-a1a5-a96536f19121 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:19.692273 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96529ed5-7604-44df-b407-4b09ce0dc6e8 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:19.709778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:19.727469 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e9d0e-b387-44a3-a54d-ca707cbc1901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:19.728840 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e9d0e-b387-44a3-a54d-ca707cbc1901 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:19.734772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e9d0e-b387-44a3-a54d-ca707cbc1901 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:19.734772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e9d0e-b387-44a3-a54d-ca707cbc1901 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:19.734965 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:19.736384 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e9d0e-b387-44a3-a54d-ca707cbc1901 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:19.737847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e9d0e-b387-44a3-a54d-ca707cbc1901 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:19.752275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-ba6e9d0e-b387-44a3-a54d-ca707cbc1901 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:19.752275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:19.752275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:19.752275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:19.752275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:19.752275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:19.752275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:19.752275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:19.752275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:19.752275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:19.752275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:19.752275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:19.752275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:19.752275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:19.752275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:19.753400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:19.754304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:19.754304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:19.754304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:19.754304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:19.754304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:19.754304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:19.754304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:19.754304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:19.754304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 319/639] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:19.755393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e9d0e-b387-44a3-a54d-ca707cbc1901 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:19.770980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ab245-1162-47fe-a1a5-a96536f19121 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:19.806348 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd3bb3-7b50-4263-aafc-edba6e66cceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:19.807592 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd3bb3-7b50-4263-aafc-edba6e66cceb None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:19.807880 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd3bb3-7b50-4263-aafc-edba6e66cceb None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:19.808154 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd3bb3-7b50-4263-aafc-edba6e66cceb None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:19.808938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd3bb3-7b50-4263-aafc-edba6e66cceb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:19.809245 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd3bb3-7b50-4263-aafc-edba6e66cceb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:19.810712 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd3bb3-7b50-4263-aafc-edba6e66cceb None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:19.823489 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 321/640] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:19.824297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd3bb3-7b50-4263-aafc-edba6e66cceb None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:19.860502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e9d0e-b387-44a3-a54d-ca707cbc1901 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:19.896187 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd708173-cdf4-4b96-ab6c-96fe8ebbd671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:19.897017 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd708173-cdf4-4b96-ab6c-96fe8ebbd671 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:19.897503 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd708173-cdf4-4b96-ab6c-96fe8ebbd671 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:19.898009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd708173-cdf4-4b96-ab6c-96fe8ebbd671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:19.913122 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd708173-cdf4-4b96-ab6c-96fe8ebbd671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:19.914512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd708173-cdf4-4b96-ab6c-96fe8ebbd671 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:19.932245 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd3bb3-7b50-4263-aafc-edba6e66cceb None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:19.936312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-bd708173-cdf4-4b96-ab6c-96fe8ebbd671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:19.936312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:19.936312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:19.936312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:19.936312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:19.936312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:19.936312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:19.936312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:19.936312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:19.936312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:19.936312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:19.936312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:19.936312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:19.936312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:19.936312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:19.937500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:19.938471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:19.938471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:19.938471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:19.938471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:19.938471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 320/641] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:19.938471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd708173-cdf4-4b96-ab6c-96fe8ebbd671 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:19.961116 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:19.970645 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ee4e8-ab15-4618-a660-9a589aaa6d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:19.971451 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ee4e8-ab15-4618-a660-9a589aaa6d15 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:19.971770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ee4e8-ab15-4618-a660-9a589aaa6d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:19.972083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ee4e8-ab15-4618-a660-9a589aaa6d15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:19.973346 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:19.983144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ee4e8-ab15-4618-a660-9a589aaa6d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:19.991367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-1f4ee4e8-ab15-4618-a660-9a589aaa6d15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:19.991367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:19.991367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:19.991367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:19.991367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:19.991367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:19.991367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:19.991367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:19.991367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:19.991367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:19.991367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:19.991367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:19.991367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:19.991367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:19.991367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:19.991367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:19.992683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:19.992683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:19.992683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:19.992683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:19.992683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:19.992683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:19.992683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:19.992683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:19.992683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:19.992683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:19.992683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:19.992683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:19.992683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:19.992683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:19.992683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:19.992683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:19.994165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:19.994165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:19.994165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:19.994165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:19.994165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:19.994165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:19.994165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:19.994165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:19.994165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:19.994165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:19.994165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:19.994165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:19.994165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 322/642] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:19.994165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ee4e8-ab15-4618-a660-9a589aaa6d15 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:20.056524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd708173-cdf4-4b96-ab6c-96fe8ebbd671 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:20.092941 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f647e-f353-4031-a24d-844468eaeca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:20.093604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f647e-f353-4031-a24d-844468eaeca0 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:20.093889 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f647e-f353-4031-a24d-844468eaeca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:20.094167 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f647e-f353-4031-a24d-844468eaeca0 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:20.094942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f647e-f353-4031-a24d-844468eaeca0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:20.095249 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f647e-f353-4031-a24d-844468eaeca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:20.096732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f647e-f353-4031-a24d-844468eaeca0 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:20.098037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ee4e8-ab15-4618-a660-9a589aaa6d15 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:20.105950 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 321/643] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:20.106793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f647e-f353-4031-a24d-844468eaeca0 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:20.135384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e93cf-3a9e-4091-a2ef-1ba875ea2c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:20.136054 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e93cf-3a9e-4091-a2ef-1ba875ea2c41 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:20.136345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e93cf-3a9e-4091-a2ef-1ba875ea2c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:20.136695 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e93cf-3a9e-4091-a2ef-1ba875ea2c41 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:20.137523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e93cf-3a9e-4091-a2ef-1ba875ea2c41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:20.137841 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e93cf-3a9e-4091-a2ef-1ba875ea2c41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:20.139424 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e93cf-3a9e-4091-a2ef-1ba875ea2c41 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:20.152494 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 323/644] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:20.153332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e93cf-3a9e-4091-a2ef-1ba875ea2c41 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:20.201049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:20.206850 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:20.216697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f647e-f353-4031-a24d-844468eaeca0 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:20.252593 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e93cf-3a9e-4091-a2ef-1ba875ea2c41 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:20.255756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a3fde-7fb0-4089-be85-f7c2564171d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:20.256416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a3fde-7fb0-4089-be85-f7c2564171d2 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:20.256702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a3fde-7fb0-4089-be85-f7c2564171d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:20.256979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a3fde-7fb0-4089-be85-f7c2564171d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:20.257809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a3fde-7fb0-4089-be85-f7c2564171d2 None admin] RBAC: Adding query filter params (role.id=29b8b950372848ffa9a75e778ee66b4c, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=7e1355d18fb049468651b3675c217d60) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:20.258123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a3fde-7fb0-4089-be85-f7c2564171d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:20.260199 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a3fde-7fb0-4089-be85-f7c2564171d2 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:20.266255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 322/645] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:20 2022] GET /identity/v3/role_assignments?user.id=7e1355d18fb049468651b3675c217d60&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=29b8b950372848ffa9a75e778ee66b4c => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:20.267316 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a3fde-7fb0-4089-be85-f7c2564171d2 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:20.304119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45acdc-94d4-4882-88c5-11f221d2d780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:20.304833 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45acdc-94d4-4882-88c5-11f221d2d780 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:20.305124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45acdc-94d4-4882-88c5-11f221d2d780 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:20.305403 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45acdc-94d4-4882-88c5-11f221d2d780 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:20.306430 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45acdc-94d4-4882-88c5-11f221d2d780 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:20.306813 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45acdc-94d4-4882-88c5-11f221d2d780 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:20.308243 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45acdc-94d4-4882-88c5-11f221d2d780 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:20.322062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 324/646] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:20.322861 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45acdc-94d4-4882-88c5-11f221d2d780 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:20.389391 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a3fde-7fb0-4089-be85-f7c2564171d2 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:20.428748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc10f11-935a-440f-8ba1-19a50354c482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:20.429465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc10f11-935a-440f-8ba1-19a50354c482 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:20.429903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc10f11-935a-440f-8ba1-19a50354c482 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:20.430199 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc10f11-935a-440f-8ba1-19a50354c482 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:20.431210 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc10f11-935a-440f-8ba1-19a50354c482 None admin] RBAC: Adding query filter params (name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:20.431654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc10f11-935a-440f-8ba1-19a50354c482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:20.433071 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc10f11-935a-440f-8ba1-19a50354c482 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:20.444813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 323/647] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:20.445790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc10f11-935a-440f-8ba1-19a50354c482 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:20.451185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45acdc-94d4-4882-88c5-11f221d2d780 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:20.459128 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:20.466333 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:20.492677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0e429-059d-46e4-b640-17e4803f5383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:20.493719 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0e429-059d-46e4-b640-17e4803f5383 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:20.494071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0e429-059d-46e4-b640-17e4803f5383 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:20.494362 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0e429-059d-46e4-b640-17e4803f5383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:20.512563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0e429-059d-46e4-b640-17e4803f5383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:20.514041 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0e429-059d-46e4-b640-17e4803f5383 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:20.536590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-30b0e429-059d-46e4-b640-17e4803f5383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:20.536590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:20.536590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:20.536590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:20.536590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:20.536590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:20.536590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:20.536590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:20.536590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:20.536590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:20.536590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:20.536590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:20.536590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:20.536590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:20.536590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:20.537799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:20.538947 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:20.538947 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:20.538947 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:20.538947 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:20.539555 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 325/648] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:20.540493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0e429-059d-46e4-b640-17e4803f5383 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:20.567402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc10f11-935a-440f-8ba1-19a50354c482 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:20.604918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a5e1e-169f-4745-a8fb-f748f19e7067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:20.606508 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a5e1e-169f-4745-a8fb-f748f19e7067 None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:20.607071 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a5e1e-169f-4745-a8fb-f748f19e7067 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:20.607836 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a5e1e-169f-4745-a8fb-f748f19e7067 None admin] PATH_INFO: `/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/f88dfe620ff04ffe88088a8a23c7d6e9/roles/29b8b950372848ffa9a75e778ee66b4c` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:20.626915 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a5e1e-169f-4745-a8fb-f748f19e7067 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f777cd6a0c744e594f8e29ab58cf8bd, user_id=f88dfe620ff04ffe88088a8a23c7d6e9, role_id=29b8b950372848ffa9a75e778ee66b4c)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:20.628532 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a5e1e-169f-4745-a8fb-f748f19e7067 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:20.651390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0e429-059d-46e4-b640-17e4803f5383 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:20.686332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a1cdc-94d1-40e5-980a-070fd330dc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:20.687268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a1cdc-94d1-40e5-980a-070fd330dc20 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:20.687558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a1cdc-94d1-40e5-980a-070fd330dc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:20.687839 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a1cdc-94d1-40e5-980a-070fd330dc20 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:20.695806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a1cdc-94d1-40e5-980a-070fd330dc20 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:20.697337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a1cdc-94d1-40e5-980a-070fd330dc20 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:20.716116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 324/649] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:53:20 2022] PUT /identity/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/f88dfe620ff04ffe88088a8a23c7d6e9/roles/29b8b950372848ffa9a75e778ee66b4c => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:53:20.716116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a5e1e-169f-4745-a8fb-f748f19e7067 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:20.716384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-aa4a1cdc-94d1-40e5-980a-070fd330dc20 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 14:53:20.716384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:20.716384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:20.716384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:20.716384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:20.716384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:20.716384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:20.716384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:20.716384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:20.716384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:20.716384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:20.716384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:20.716384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:20.716384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:20.716384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:20.716384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:20.717367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:20.718399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:20.718399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:20.718399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:20.718399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:20.718399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:20.718399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:20.718399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:20.718399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:20.718399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 14:53:20.718399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:20.718399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 326/650] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:53:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:20.718399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a1cdc-94d1-40e5-980a-070fd330dc20 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:20.767009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:20.804639 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:20.846200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a5e1e-169f-4745-a8fb-f748f19e7067 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:20.874211 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a1cdc-94d1-40e5-980a-070fd330dc20 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:20.910746 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7af82f-22a5-4073-8079-d77996dc3381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:20.912310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7af82f-22a5-4073-8079-d77996dc3381 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:20.912812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7af82f-22a5-4073-8079-d77996dc3381 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:20.913302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7af82f-22a5-4073-8079-d77996dc3381 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:20.925630 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea9713-a083-4f39-ad81-3998d64c976e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:20.926246 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7af82f-22a5-4073-8079-d77996dc3381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:20.926364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea9713-a083-4f39-ad81-3998d64c976e None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:20.926624 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea9713-a083-4f39-ad81-3998d64c976e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:20.926915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea9713-a083-4f39-ad81-3998d64c976e None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:20.928096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea9713-a083-4f39-ad81-3998d64c976e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:20.928096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea9713-a083-4f39-ad81-3998d64c976e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:20.929374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea9713-a083-4f39-ad81-3998d64c976e None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:20.929672 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7af82f-22a5-4073-8079-d77996dc3381 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:20.943739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 327/651] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:20.944989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea9713-a083-4f39-ad81-3998d64c976e None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:20.958137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-fb7af82f-22a5-4073-8079-d77996dc3381 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:20.958137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:20.958137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:20.958137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:20.958137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:20.958137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:20.958137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:20.958137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:20.958137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:20.958137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:20.958137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:20.958137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:20.958137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:20.958137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:20.958137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:20.959423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:20.960341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:20.960341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:20.960341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:20.960341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:20.960341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 325/652] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:20.960632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7af82f-22a5-4073-8079-d77996dc3381 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:21.045065 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:21.070199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea9713-a083-4f39-ad81-3998d64c976e None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:21.103149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7af82f-22a5-4073-8079-d77996dc3381 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:21.123606 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:21.127683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46524ad0-cfba-4def-ae99-db0466ff5e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:21.128379 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46524ad0-cfba-4def-ae99-db0466ff5e7b None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:21.128673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46524ad0-cfba-4def-ae99-db0466ff5e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:21.128969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46524ad0-cfba-4def-ae99-db0466ff5e7b None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:21.130130 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46524ad0-cfba-4def-ae99-db0466ff5e7b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:21.130454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46524ad0-cfba-4def-ae99-db0466ff5e7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:21.132114 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46524ad0-cfba-4def-ae99-db0466ff5e7b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:21.145700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b07c9e-c01e-4b18-86bf-d33682058e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:21.146945 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b07c9e-c01e-4b18-86bf-d33682058e36 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:21.147462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b07c9e-c01e-4b18-86bf-d33682058e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:21.147989 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b07c9e-c01e-4b18-86bf-d33682058e36 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:21.157186 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b07c9e-c01e-4b18-86bf-d33682058e36 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:21.158719 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 328/653] 10.222.0.83 () {56 vars in 1137 bytes} [Mon Jul 18 14:53:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:21.159524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b07c9e-c01e-4b18-86bf-d33682058e36 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:21.159735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46524ad0-cfba-4def-ae99-db0466ff5e7b None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:21.182190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-a7b07c9e-c01e-4b18-86bf-d33682058e36 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 14:53:21.182190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:21.182190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:21.182190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:21.182190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:21.182190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:21.182190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:21.182190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:21.182190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:21.182190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:21.182190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:21.182190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:21.182190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:21.182190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:21.182190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:21.182190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:21.183255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:21.184340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:21.184340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:21.184340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:21.184340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:21.184340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:21.184340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:21.184340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:21.184340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:21.184340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 14:53:21.184340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:21.186726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 326/654] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:53:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:21.187931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b07c9e-c01e-4b18-86bf-d33682058e36 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:21.290892 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46524ad0-cfba-4def-ae99-db0466ff5e7b None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:21.304115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b07c9e-c01e-4b18-86bf-d33682058e36 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:21.330452 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94784346-9d7f-4bfb-ba0a-94a5c394b8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:21.331493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94784346-9d7f-4bfb-ba0a-94a5c394b8df None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:21.332261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94784346-9d7f-4bfb-ba0a-94a5c394b8df None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:21.332781 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94784346-9d7f-4bfb-ba0a-94a5c394b8df None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:21.333744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94784346-9d7f-4bfb-ba0a-94a5c394b8df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:21.334329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94784346-9d7f-4bfb-ba0a-94a5c394b8df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:21.336188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94784346-9d7f-4bfb-ba0a-94a5c394b8df None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:21.340100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:21.350438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462ee3d-e187-440e-94a7-e1e72c56d8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:21.351375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462ee3d-e187-440e-94a7-e1e72c56d8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:21.351375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462ee3d-e187-440e-94a7-e1e72c56d8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:21.351669 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462ee3d-e187-440e-94a7-e1e72c56d8d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:21.352427 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 329/655] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:21.353749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94784346-9d7f-4bfb-ba0a-94a5c394b8df None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:21.412362 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:21.438732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462ee3d-e187-440e-94a7-e1e72c56d8d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:21.440657 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462ee3d-e187-440e-94a7-e1e72c56d8d9 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:21.485155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94784346-9d7f-4bfb-ba0a-94a5c394b8df None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:21.525897 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d5101-d6b1-4d8c-927e-72cad73fb98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:21.527077 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d5101-d6b1-4d8c-927e-72cad73fb98b None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:21.527607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d5101-d6b1-4d8c-927e-72cad73fb98b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:21.528117 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d5101-d6b1-4d8c-927e-72cad73fb98b None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:21.529255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d5101-d6b1-4d8c-927e-72cad73fb98b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:21.529785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d5101-d6b1-4d8c-927e-72cad73fb98b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:21.531507 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d5101-d6b1-4d8c-927e-72cad73fb98b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:21.552717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 330/656] 10.222.0.83 () {56 vars in 1137 bytes} [Mon Jul 18 14:53:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:21.554048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d5101-d6b1-4d8c-927e-72cad73fb98b None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:21.582698 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 327/657] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:53:21.583908 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462ee3d-e187-440e-94a7-e1e72c56d8d9 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:21.656496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:21.672275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:21.698039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462ee3d-e187-440e-94a7-e1e72c56d8d9 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:21.737313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad50d5-8e9f-416e-a4c7-2a471e8bef5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:21.738730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad50d5-8e9f-416e-a4c7-2a471e8bef5b None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:21.739576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad50d5-8e9f-416e-a4c7-2a471e8bef5b None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:21.740075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad50d5-8e9f-416e-a4c7-2a471e8bef5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:21.748127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d5101-d6b1-4d8c-927e-72cad73fb98b None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:21.811099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269f69d-9308-414a-b5bb-a9629618fe7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:21.812777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269f69d-9308-414a-b5bb-a9629618fe7d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:21.813340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269f69d-9308-414a-b5bb-a9629618fe7d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:21.813785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269f69d-9308-414a-b5bb-a9629618fe7d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:21.826953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad50d5-8e9f-416e-a4c7-2a471e8bef5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:21.828835 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269f69d-9308-414a-b5bb-a9629618fe7d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:21.829063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad50d5-8e9f-416e-a4c7-2a471e8bef5b None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:21.831640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269f69d-9308-414a-b5bb-a9629618fe7d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:21.861859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-3269f69d-9308-414a-b5bb-a9629618fe7d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 14:53:21.861859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:21.861859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:21.861859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:21.861859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:21.861859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:21.861859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:21.861859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:21.861859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:21.861859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:21.861859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:21.861859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:21.861859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:21.861859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:21.861859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:21.861859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:21.862923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:21.863972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:21.863972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:21.863972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:21.863972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:21.863972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:21.863972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:21.863972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:21.863972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:21.863972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 14:53:21.863972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:21.871051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 331/658] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:21.872702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269f69d-9308-414a-b5bb-a9629618fe7d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:21.992091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:21.999376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 328/659] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:53:22.001129 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad50d5-8e9f-416e-a4c7-2a471e8bef5b None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:22.003840 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269f69d-9308-414a-b5bb-a9629618fe7d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:22.030634 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:22.063503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79eeb84-1263-4156-8de5-4b81cd191f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:22.064936 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79eeb84-1263-4156-8de5-4b81cd191f02 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:22.065447 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79eeb84-1263-4156-8de5-4b81cd191f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:22.065934 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79eeb84-1263-4156-8de5-4b81cd191f02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:22.082176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79eeb84-1263-4156-8de5-4b81cd191f02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:22.084364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79eeb84-1263-4156-8de5-4b81cd191f02 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:22.113435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-c79eeb84-1263-4156-8de5-4b81cd191f02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:22.113435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:22.113435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:22.113435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:22.113435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:22.113435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:22.113435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:22.113435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:22.113435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:22.113435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:22.113435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:22.113435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:22.113435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:22.113435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:22.113435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:22.115590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:22.116783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:22.116783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:22.116783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:22.116783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:22.122081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 332/660] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:22.123221 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79eeb84-1263-4156-8de5-4b81cd191f02 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:22.126359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad50d5-8e9f-416e-a4c7-2a471e8bef5b None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:22.165944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321b488-5995-49f2-99fe-018786f2c129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:22.167156 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321b488-5995-49f2-99fe-018786f2c129 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:22.167589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321b488-5995-49f2-99fe-018786f2c129 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:22.168014 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321b488-5995-49f2-99fe-018786f2c129 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:22.168995 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321b488-5995-49f2-99fe-018786f2c129 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:22.169451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321b488-5995-49f2-99fe-018786f2c129 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:22.171123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321b488-5995-49f2-99fe-018786f2c129 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:22.194689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 329/661] 10.222.0.83 () {56 vars in 1139 bytes} [Mon Jul 18 14:53:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:22.198649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321b488-5995-49f2-99fe-018786f2c129 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:22.237639 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:22.269497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79eeb84-1263-4156-8de5-4b81cd191f02 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:22.308860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321b488-5995-49f2-99fe-018786f2c129 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:22.328205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64602f3-c78b-4647-9494-c184045bb8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:22.329654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64602f3-c78b-4647-9494-c184045bb8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:22.330256 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64602f3-c78b-4647-9494-c184045bb8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:22.330987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64602f3-c78b-4647-9494-c184045bb8c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:22.346121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8d9a1-b772-415a-b4b5-0d8939e91f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:22.346849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8d9a1-b772-415a-b4b5-0d8939e91f50 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:22.347205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8d9a1-b772-415a-b4b5-0d8939e91f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:22.347396 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8d9a1-b772-415a-b4b5-0d8939e91f50 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:22.348124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8d9a1-b772-415a-b4b5-0d8939e91f50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:22.348426 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8d9a1-b772-415a-b4b5-0d8939e91f50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:22.349210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64602f3-c78b-4647-9494-c184045bb8c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:22.349826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8d9a1-b772-415a-b4b5-0d8939e91f50 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:22.351937 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64602f3-c78b-4647-9494-c184045bb8c8 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:22.369717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 330/662] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:22.369717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8d9a1-b772-415a-b4b5-0d8939e91f50 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:22.390704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-e64602f3-c78b-4647-9494-c184045bb8c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:22.390704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:22.390704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:22.390704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:22.390704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:22.390704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:22.390704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:22.390704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:22.390704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:22.390704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:22.390704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:22.390704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:22.390704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:22.390704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:22.390704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:22.392415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:22.394019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:22.394019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:22.394019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:22.394019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:22.397011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 333/663] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:22.398836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64602f3-c78b-4647-9494-c184045bb8c8 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:22.432703 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:22.471739 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:22.478776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8d9a1-b772-415a-b4b5-0d8939e91f50 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:22.517277 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76dbf0-9e72-49f7-a276-56478e14677d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:22.517969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76dbf0-9e72-49f7-a276-56478e14677d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:22.518253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76dbf0-9e72-49f7-a276-56478e14677d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:22.518529 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76dbf0-9e72-49f7-a276-56478e14677d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:22.561756 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64602f3-c78b-4647-9494-c184045bb8c8 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:22.595924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76dbf0-9e72-49f7-a276-56478e14677d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:22.597473 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76dbf0-9e72-49f7-a276-56478e14677d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:22.612638 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf13de8-7b6f-4cfd-934c-9460194c3612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:22.614195 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf13de8-7b6f-4cfd-934c-9460194c3612 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:22.614831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf13de8-7b6f-4cfd-934c-9460194c3612 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:22.615444 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf13de8-7b6f-4cfd-934c-9460194c3612 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:22.626420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf13de8-7b6f-4cfd-934c-9460194c3612 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:22.629208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf13de8-7b6f-4cfd-934c-9460194c3612 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:22.650900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-6cf13de8-7b6f-4cfd-934c-9460194c3612 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:22.650900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:22.650900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:22.650900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:22.650900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:22.650900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:22.650900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:22.650900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:22.650900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:22.650900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:22.650900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:22.650900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:22.650900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:22.650900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:22.650900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:22.652016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:22.653019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:22.653019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:22.653019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:22.653019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:22.653019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:22.653019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:22.653019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:22.653019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:22.659822 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 334/664] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:22.660968 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf13de8-7b6f-4cfd-934c-9460194c3612 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:22.706928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:22.721880 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 331/665] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:53:22.723370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76dbf0-9e72-49f7-a276-56478e14677d None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:22.781259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:22.819156 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf13de8-7b6f-4cfd-934c-9460194c3612 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:22.841338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76dbf0-9e72-49f7-a276-56478e14677d None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:22.867909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ece97-8895-44e2-b97c-71e433981219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:22.868801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ece97-8895-44e2-b97c-71e433981219 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:22.868945 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ece97-8895-44e2-b97c-71e433981219 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:22.869230 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ece97-8895-44e2-b97c-71e433981219 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:22.869962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ece97-8895-44e2-b97c-71e433981219 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:22.870274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ece97-8895-44e2-b97c-71e433981219 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:22.871698 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ece97-8895-44e2-b97c-71e433981219 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:22.883803 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 335/666] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:22.887925 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee8a37-46f1-49db-a848-93d8b1d05b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:22.889121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee8a37-46f1-49db-a848-93d8b1d05b68 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:22.889292 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ece97-8895-44e2-b97c-71e433981219 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:22.889722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee8a37-46f1-49db-a848-93d8b1d05b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:22.889943 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee8a37-46f1-49db-a848-93d8b1d05b68 None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:22.891322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee8a37-46f1-49db-a848-93d8b1d05b68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:22.891782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee8a37-46f1-49db-a848-93d8b1d05b68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:22.893970 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee8a37-46f1-49db-a848-93d8b1d05b68 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:22.911704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 332/667] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:22.913375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d321c7e-1126-415e-a5c6-fbdd1f14313f None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:22.913898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d321c7e-1126-415e-a5c6-fbdd1f14313f None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:22.914393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d321c7e-1126-415e-a5c6-fbdd1f14313f None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:22.916810 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 333/668] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:22.918052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d321c7e-1126-415e-a5c6-fbdd1f14313f None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:22.995465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:23.034380 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ece97-8895-44e2-b97c-71e433981219 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:23.045875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d321c7e-1126-415e-a5c6-fbdd1f14313f None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:23.073354 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6b2f7-4cd4-48e5-ad9c-b43c651d630a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:23.074139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6b2f7-4cd4-48e5-ad9c-b43c651d630a None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:23.074422 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6b2f7-4cd4-48e5-ad9c-b43c651d630a None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:23.074743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6b2f7-4cd4-48e5-ad9c-b43c651d630a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:23.081432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:23.081592 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552c8cd-2725-46d8-98f2-fbf6ff30e707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:23.082389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552c8cd-2725-46d8-98f2-fbf6ff30e707 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:23.082554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552c8cd-2725-46d8-98f2-fbf6ff30e707 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:23.082819 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552c8cd-2725-46d8-98f2-fbf6ff30e707 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:23.085520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6b2f7-4cd4-48e5-ad9c-b43c651d630a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:23.087656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6b2f7-4cd4-48e5-ad9c-b43c651d630a None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:23.090430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552c8cd-2725-46d8-98f2-fbf6ff30e707 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:23.097434 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552c8cd-2725-46d8-98f2-fbf6ff30e707 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:23.106475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-69d6b2f7-4cd4-48e5-ad9c-b43c651d630a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:23.106475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:23.106475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:23.106475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:23.106475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:23.106475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:23.106475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:23.106475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:23.106475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:23.106475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:23.106475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:23.106475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:23.106475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:23.106475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:23.106475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:23.107683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:23.108603 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:23.108603 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:23.108603 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:23.108603 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:23.108830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-0552c8cd-2725-46d8-98f2-fbf6ff30e707 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:23.108830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:23.108830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:23.108830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:23.108830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:23.108830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:23.108830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:23.108830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:23.108830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:23.108830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:23.108830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:23.108830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:23.108830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:23.108830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:23.108830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:23.111415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:23.112341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:23.112341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:23.112341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:23.112341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:23.112341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:23.112341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:23.112341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:23.112341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:23.112779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 336/669] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:23.112852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 334/670] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:23.112852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee8fcb-ec6c-4ef6-b271-5af720f4a099 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:23.112852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee8fcb-ec6c-4ef6-b271-5af720f4a099 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:23.112852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee8fcb-ec6c-4ef6-b271-5af720f4a099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:23.113446 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6b2f7-4cd4-48e5-ad9c-b43c651d630a None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:23.129293 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-70ee8fcb-ec6c-4ef6-b271-5af720f4a099 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:23.130525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-70ee8fcb-ec6c-4ef6-b271-5af720f4a099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:23.201843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 335/671] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:23.203171 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b40408-5b65-40cf-a22b-d0a3bc3af8f5 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:23.203548 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b40408-5b65-40cf-a22b-d0a3bc3af8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:23.203834 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b40408-5b65-40cf-a22b-d0a3bc3af8f5 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:23.205186 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 336/672] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:23.205874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b40408-5b65-40cf-a22b-d0a3bc3af8f5 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:23.240347 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6b2f7-4cd4-48e5-ad9c-b43c651d630a None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:23.260851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:23.278607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f44ba-ed50-4099-a761-8f7b25b1c0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:23.279276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f44ba-ed50-4099-a761-8f7b25b1c0aa None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:23.279535 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f44ba-ed50-4099-a761-8f7b25b1c0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:23.279813 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f44ba-ed50-4099-a761-8f7b25b1c0aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:23.280708 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f44ba-ed50-4099-a761-8f7b25b1c0aa None admin] RBAC: Adding query filter params (role.id=29b8b950372848ffa9a75e778ee66b4c, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=ef8dcb0a0f2c4a2d955544324213739c) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:23.281055 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f44ba-ed50-4099-a761-8f7b25b1c0aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:23.282491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f44ba-ed50-4099-a761-8f7b25b1c0aa None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:23.288705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 337/673] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:23 2022] GET /identity/v3/role_assignments?user.id=ef8dcb0a0f2c4a2d955544324213739c&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=29b8b950372848ffa9a75e778ee66b4c => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:23.289557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f44ba-ed50-4099-a761-8f7b25b1c0aa None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:23.335805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b40408-5b65-40cf-a22b-d0a3bc3af8f5 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:23.364204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:23.384510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980847f-f5a6-4810-8c8d-4369b5b9dcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:23.385463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980847f-f5a6-4810-8c8d-4369b5b9dcb5 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:23.385869 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980847f-f5a6-4810-8c8d-4369b5b9dcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:23.386266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980847f-f5a6-4810-8c8d-4369b5b9dcb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:23.387402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980847f-f5a6-4810-8c8d-4369b5b9dcb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:23.387866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980847f-f5a6-4810-8c8d-4369b5b9dcb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:23.390106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980847f-f5a6-4810-8c8d-4369b5b9dcb5 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:23.413537 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 337/674] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:23.413537 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1179385-b569-47ac-9e24-518c94b2cbdd None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:23.413537 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1179385-b569-47ac-9e24-518c94b2cbdd None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:23.413537 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1179385-b569-47ac-9e24-518c94b2cbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:23.425184 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f44ba-ed50-4099-a761-8f7b25b1c0aa None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:23.436166 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-f1179385-b569-47ac-9e24-518c94b2cbdd None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:23.437915 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f1179385-b569-47ac-9e24-518c94b2cbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:23.469966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c180c-8a03-4b14-8a9a-d081659e570a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:23.470615 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c180c-8a03-4b14-8a9a-d081659e570a None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:23.470930 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c180c-8a03-4b14-8a9a-d081659e570a None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:23.471218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c180c-8a03-4b14-8a9a-d081659e570a None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:23.472014 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c180c-8a03-4b14-8a9a-d081659e570a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:23.472325 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c180c-8a03-4b14-8a9a-d081659e570a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:23.473779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c180c-8a03-4b14-8a9a-d081659e570a None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:23.490478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 338/675] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:23.491959 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c180c-8a03-4b14-8a9a-d081659e570a None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:23.526191 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 338/676] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:23.528015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1179385-b569-47ac-9e24-518c94b2cbdd None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:23.578105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:23.622333 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c180c-8a03-4b14-8a9a-d081659e570a None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:23.639079 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:23.657964 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1179385-b569-47ac-9e24-518c94b2cbdd None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:23.663509 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd43874-fb04-43d2-887e-6f73018025df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:23.664311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd43874-fb04-43d2-887e-6f73018025df None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:23.664601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd43874-fb04-43d2-887e-6f73018025df None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:23.665004 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd43874-fb04-43d2-887e-6f73018025df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:23.676913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd43874-fb04-43d2-887e-6f73018025df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:23.686083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-cfd43874-fb04-43d2-887e-6f73018025df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:53:23.686083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:23.686083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:23.686083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:23.686083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:23.686083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:23.686083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:23.686083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:23.686083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:23.686083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:23.686083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:23.686083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:23.686083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:23.686083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:23.686083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:23.686083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:23.687316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:23.687316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:23.687316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:23.687316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:23.687316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:23.687316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:23.687316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:23.687316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:23.687316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:23.687316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:23.687316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:23.687316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:23.687316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:23.687316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:23.687316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:23.687316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:23.688218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:23.688218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:23.688218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:23.688218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:23.688218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:23.688218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:23.688218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:23.688218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:23.688218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:23.688218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:23.688218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:53:23.688218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:23.691370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 339/677] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:53:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:23.692011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd43874-fb04-43d2-887e-6f73018025df None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:23.707011 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefa1e9-c320-4c8f-b837-3e9e9b4546dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:23.707824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefa1e9-c320-4c8f-b837-3e9e9b4546dc None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:23.708115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefa1e9-c320-4c8f-b837-3e9e9b4546dc None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:23.708398 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefa1e9-c320-4c8f-b837-3e9e9b4546dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:23.717771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcefa1e9-c320-4c8f-b837-3e9e9b4546dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:23.721379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcefa1e9-c320-4c8f-b837-3e9e9b4546dc None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:23.736227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-fcefa1e9-c320-4c8f-b837-3e9e9b4546dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:23.736227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:23.736227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:23.736227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:23.736227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:23.736227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:23.736227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:23.736227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:23.736227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:23.736227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:23.736227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:23.736227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:23.736227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:23.736227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:23.736227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:23.737403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:23.744712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:23.744712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:23.744712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:23.744712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:23.744712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:23.744712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:23.744712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:23.744712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:23.744712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 339/678] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:23.744712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefa1e9-c320-4c8f-b837-3e9e9b4546dc None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:23.818279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd43874-fb04-43d2-887e-6f73018025df None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:23.852145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:23.858973 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefa1e9-c320-4c8f-b837-3e9e9b4546dc None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:23.863118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef8b7a-fab7-48f9-8ef9-75b942484224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:23.863792 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef8b7a-fab7-48f9-8ef9-75b942484224 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:23.864086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef8b7a-fab7-48f9-8ef9-75b942484224 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:23.864368 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef8b7a-fab7-48f9-8ef9-75b942484224 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:23.865218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef8b7a-fab7-48f9-8ef9-75b942484224 None admin] RBAC: Adding query filter params (role.id=29b8b950372848ffa9a75e778ee66b4c, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=9218503bc8a647d68ffab6945bae456a) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:23.867560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef8b7a-fab7-48f9-8ef9-75b942484224 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:23.867560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef8b7a-fab7-48f9-8ef9-75b942484224 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:23.875919 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 340/679] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:23 2022] GET /identity/v3/role_assignments?user.id=9218503bc8a647d68ffab6945bae456a&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=29b8b950372848ffa9a75e778ee66b4c => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:23.877119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef8b7a-fab7-48f9-8ef9-75b942484224 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:23.901106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502f7aa-a04d-4d81-be0b-9cc9120abded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:23.902038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502f7aa-a04d-4d81-be0b-9cc9120abded None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:23.902397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502f7aa-a04d-4d81-be0b-9cc9120abded None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:23.902792 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502f7aa-a04d-4d81-be0b-9cc9120abded None admin] PATH_INFO: `/v3/roles/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:23.916808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502f7aa-a04d-4d81-be0b-9cc9120abded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:23.928254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-1502f7aa-a04d-4d81-be0b-9cc9120abded None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:23.928254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:23.928254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:23.928254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:23.928254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:23.928254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:23.928254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:23.928254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:23.928254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:23.928254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:23.928254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:23.928254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:23.928254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:23.928254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:23.928254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:23.928254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:23.929875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:23.929875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:23.929875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:23.929875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:23.929875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:23.929875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:23.929875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:23.929875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:23.929875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:23.929875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:23.929875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:23.929875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:23.929875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:23.929875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:23.929875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:23.929875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:23.932080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:23.932080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:23.932080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:23.932080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:23.932080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:23.932080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:23.932080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:23.932080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:23.932080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:23.932080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:23.932080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:23.932080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:23.932080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 340/680] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:23.932080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502f7aa-a04d-4d81-be0b-9cc9120abded None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:23.968277 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:24.076164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef8b7a-fab7-48f9-8ef9-75b942484224 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:24.076412 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502f7aa-a04d-4d81-be0b-9cc9120abded None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:24.125391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8abdd8-89e2-4257-870b-d4adf82953ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:24.126073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8abdd8-89e2-4257-870b-d4adf82953ee None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:24.126358 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8abdd8-89e2-4257-870b-d4adf82953ee None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:24.126746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8abdd8-89e2-4257-870b-d4adf82953ee None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:24.127515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8abdd8-89e2-4257-870b-d4adf82953ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:24.127851 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8abdd8-89e2-4257-870b-d4adf82953ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:24.129236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8abdd8-89e2-4257-870b-d4adf82953ee None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:24.131845 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32472749-6a2b-4230-9e09-452e65013f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:24.133123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32472749-6a2b-4230-9e09-452e65013f02 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:24.133638 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32472749-6a2b-4230-9e09-452e65013f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:24.134625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32472749-6a2b-4230-9e09-452e65013f02 None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:24.135816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32472749-6a2b-4230-9e09-452e65013f02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:24.136792 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32472749-6a2b-4230-9e09-452e65013f02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:24.138800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32472749-6a2b-4230-9e09-452e65013f02 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:24.141500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 341/681] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:53:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:24.141758 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8abdd8-89e2-4257-870b-d4adf82953ee None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:24.162565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 341/682] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:24.162565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32472749-6a2b-4230-9e09-452e65013f02 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:24.192226 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:24.288249 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:24.296286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8abdd8-89e2-4257-870b-d4adf82953ee None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:24.302037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32472749-6a2b-4230-9e09-452e65013f02 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:24.347162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1f6ce-db10-4f6a-a27a-67d3f389846b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:24.348173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1f6ce-db10-4f6a-a27a-67d3f389846b None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:24.348573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1f6ce-db10-4f6a-a27a-67d3f389846b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:24.349019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1f6ce-db10-4f6a-a27a-67d3f389846b None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:24.349882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1f6ce-db10-4f6a-a27a-67d3f389846b None admin] RBAC: Adding query filter params (name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:24.350296 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1f6ce-db10-4f6a-a27a-67d3f389846b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:24.353630 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1f6ce-db10-4f6a-a27a-67d3f389846b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:24.361356 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8294ba-3a33-4631-a54d-861442e2e6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:24.362648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8294ba-3a33-4631-a54d-861442e2e6aa None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:24.363053 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 342/683] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:24.363178 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8294ba-3a33-4631-a54d-861442e2e6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:24.363478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8294ba-3a33-4631-a54d-861442e2e6aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:24.364257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1f6ce-db10-4f6a-a27a-67d3f389846b None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:24.390115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8294ba-3a33-4631-a54d-861442e2e6aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:24.392296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8294ba-3a33-4631-a54d-861442e2e6aa None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:24.418024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-dc8294ba-3a33-4631-a54d-861442e2e6aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:24.418024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:24.418024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:24.418024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:24.418024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:24.418024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:24.418024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:24.418024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:24.418024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:24.418024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:24.418024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:24.418024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:24.418024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:24.418024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:24.418024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:24.419535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:24.420540 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:24.420540 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:24.420540 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:24.420540 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:24.420540 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 342/684] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:24.420540 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8294ba-3a33-4631-a54d-861442e2e6aa None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:24.507715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1f6ce-db10-4f6a-a27a-67d3f389846b None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:24.508277 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:24.546254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8294ba-3a33-4631-a54d-861442e2e6aa None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:24.548206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506cf46-14fc-4861-90e1-33c9d286edd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:24.549212 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506cf46-14fc-4861-90e1-33c9d286edd4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:24.549501 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506cf46-14fc-4861-90e1-33c9d286edd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:24.549770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506cf46-14fc-4861-90e1-33c9d286edd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:24.551103 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506cf46-14fc-4861-90e1-33c9d286edd4 None admin] RBAC: Adding query filter params (role.id=29b8b950372848ffa9a75e778ee66b4c, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=fd86979e0f0d4a4e9921ee5704901f74) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:24.551103 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506cf46-14fc-4861-90e1-33c9d286edd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:24.552529 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506cf46-14fc-4861-90e1-33c9d286edd4 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:24.566578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 343/685] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:24 2022] GET /identity/v3/role_assignments?user.id=fd86979e0f0d4a4e9921ee5704901f74&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=29b8b950372848ffa9a75e778ee66b4c => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:24.566578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506cf46-14fc-4861-90e1-33c9d286edd4 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:24.635094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21298b6c-5f4f-4c35-a20f-895b81f59023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:24.635536 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21298b6c-5f4f-4c35-a20f-895b81f59023 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:24.635817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21298b6c-5f4f-4c35-a20f-895b81f59023 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:24.636125 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21298b6c-5f4f-4c35-a20f-895b81f59023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:24.637268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:24.670352 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21298b6c-5f4f-4c35-a20f-895b81f59023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:24.671899 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21298b6c-5f4f-4c35-a20f-895b81f59023 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:24.715814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-21298b6c-5f4f-4c35-a20f-895b81f59023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:24.715814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:24.715814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:24.715814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:24.715814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:24.715814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:24.715814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:24.715814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:24.715814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:24.715814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:24.715814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:24.715814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:24.715814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:24.715814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:24.715814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:24.716811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:24.718205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:24.718205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:24.718205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:24.718205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:24.718205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 343/686] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:24.729901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21298b6c-5f4f-4c35-a20f-895b81f59023 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:24.780215 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506cf46-14fc-4861-90e1-33c9d286edd4 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:24.832354 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba71fb-2e67-45e9-8a15-a79fc3abe6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:24.833456 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba71fb-2e67-45e9-8a15-a79fc3abe6aa None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:24.834107 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba71fb-2e67-45e9-8a15-a79fc3abe6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:24.834631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba71fb-2e67-45e9-8a15-a79fc3abe6aa None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:24.837199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba71fb-2e67-45e9-8a15-a79fc3abe6aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:24.837764 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba71fb-2e67-45e9-8a15-a79fc3abe6aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:24.839857 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba71fb-2e67-45e9-8a15-a79fc3abe6aa None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:24.855015 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 344/687] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:24.860983 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba71fb-2e67-45e9-8a15-a79fc3abe6aa None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:24.885487 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21298b6c-5f4f-4c35-a20f-895b81f59023 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:24.916408 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:24.946666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f40dd-3248-4ff8-af9f-a44ab7ae541d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:24.948644 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f40dd-3248-4ff8-af9f-a44ab7ae541d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:24.949584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f40dd-3248-4ff8-af9f-a44ab7ae541d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:24.951093 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f40dd-3248-4ff8-af9f-a44ab7ae541d None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:24.953733 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f40dd-3248-4ff8-af9f-a44ab7ae541d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:24.953733 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f40dd-3248-4ff8-af9f-a44ab7ae541d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:24.956370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f40dd-3248-4ff8-af9f-a44ab7ae541d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:24.978314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 344/688] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:24.987072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f40dd-3248-4ff8-af9f-a44ab7ae541d None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:25.000405 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:25.021654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba71fb-2e67-45e9-8a15-a79fc3abe6aa None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:25.083699 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c2dee-57e8-4e1f-b161-c400cb58f71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:25.084800 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c2dee-57e8-4e1f-b161-c400cb58f71c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:25.086452 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c2dee-57e8-4e1f-b161-c400cb58f71c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:25.086940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c2dee-57e8-4e1f-b161-c400cb58f71c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:25.101756 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c2dee-57e8-4e1f-b161-c400cb58f71c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:25.103576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c2dee-57e8-4e1f-b161-c400cb58f71c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:25.133145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-835c2dee-57e8-4e1f-b161-c400cb58f71c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:53:25.133145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:25.133145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:25.133145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:25.133145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:25.133145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:25.133145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:25.133145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:25.133145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:25.133145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:25.133145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:25.133145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:25.133145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:25.133145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:25.133145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:25.133145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:25.134667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:25.136152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:25.136152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:25.136152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:25.136152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:25.136152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:25.136152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:25.136152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:25.136152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:25.136152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:53:25.136152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:25.141502 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 345/689] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Jul 18 14:53:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:25.145577 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c2dee-57e8-4e1f-b161-c400cb58f71c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:25.148245 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f40dd-3248-4ff8-af9f-a44ab7ae541d None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:25.219229 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8a77a-b9ab-426f-8d83-6f6e9b6eb1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:25.220046 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8a77a-b9ab-426f-8d83-6f6e9b6eb1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:25.220330 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8a77a-b9ab-426f-8d83-6f6e9b6eb1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:25.220607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8a77a-b9ab-426f-8d83-6f6e9b6eb1f0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:25.236182 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8a77a-b9ab-426f-8d83-6f6e9b6eb1f0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:25.237676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8a77a-b9ab-426f-8d83-6f6e9b6eb1f0 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:25.273260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-c1e8a77a-b9ab-426f-8d83-6f6e9b6eb1f0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:53:25.273260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:25.273260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:25.273260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:25.273260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:25.273260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:25.273260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:25.273260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:25.273260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:25.273260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:25.273260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:25.273260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:25.273260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:25.273260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:25.273260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:25.273260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:25.274852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:25.276334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:25.276334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:25.276334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:25.276334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:25.276334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:25.276334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:25.276334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:25.276334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:25.276334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:53:25.276334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:25.276334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 345/690] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:53:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:25.281116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8a77a-b9ab-426f-8d83-6f6e9b6eb1f0 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:25.350484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c2dee-57e8-4e1f-b161-c400cb58f71c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:25.365592 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:25.418811 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04964408-7fa4-4e28-aad8-ba234bc30a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:25.419397 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04964408-7fa4-4e28-aad8-ba234bc30a08 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:25.419682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04964408-7fa4-4e28-aad8-ba234bc30a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:25.419963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04964408-7fa4-4e28-aad8-ba234bc30a08 None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:25.420810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04964408-7fa4-4e28-aad8-ba234bc30a08 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:25.421127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04964408-7fa4-4e28-aad8-ba234bc30a08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:25.422587 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04964408-7fa4-4e28-aad8-ba234bc30a08 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:25.460873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 346/691] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Jul 18 14:53:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:25.471471 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04964408-7fa4-4e28-aad8-ba234bc30a08 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:25.476374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:25.488460 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8a77a-b9ab-426f-8d83-6f6e9b6eb1f0 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:25.560531 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21245df5-0922-4ae7-9faf-4df71ae4b712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:25.561206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21245df5-0922-4ae7-9faf-4df71ae4b712 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:25.561489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21245df5-0922-4ae7-9faf-4df71ae4b712 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:25.561763 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21245df5-0922-4ae7-9faf-4df71ae4b712 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:25.562627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21245df5-0922-4ae7-9faf-4df71ae4b712 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:25.562937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21245df5-0922-4ae7-9faf-4df71ae4b712 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:25.565409 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21245df5-0922-4ae7-9faf-4df71ae4b712 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:25.599627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 346/692] 10.222.0.83 () {56 vars in 1135 bytes} [Mon Jul 18 14:53:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:25.603394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21245df5-0922-4ae7-9faf-4df71ae4b712 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:25.655469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04964408-7fa4-4e28-aad8-ba234bc30a08 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:25.692769 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb93ed9-6169-4549-867e-f70d1eed6481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:25.693339 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb93ed9-6169-4549-867e-f70d1eed6481 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:25.693624 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb93ed9-6169-4549-867e-f70d1eed6481 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:25.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb93ed9-6169-4549-867e-f70d1eed6481 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:25.727180 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:25.734215 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21245df5-0922-4ae7-9faf-4df71ae4b712 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:25.774819 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba520b9d-1e71-4ded-8ba8-a50cfb8cb38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:25.775511 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba520b9d-1e71-4ded-8ba8-a50cfb8cb38a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:25.775864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb93ed9-6169-4549-867e-f70d1eed6481 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:25.775988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba520b9d-1e71-4ded-8ba8-a50cfb8cb38a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:25.776165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba520b9d-1e71-4ded-8ba8-a50cfb8cb38a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:25.777483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb93ed9-6169-4549-867e-f70d1eed6481 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:25.780955 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:25.862316 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba520b9d-1e71-4ded-8ba8-a50cfb8cb38a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:25.864446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba520b9d-1e71-4ded-8ba8-a50cfb8cb38a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:25.901413 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 347/693] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 14:53:25.903094 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa33a8-436b-4dca-a2b9-af9f0091a6ed None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:25.903509 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa33a8-436b-4dca-a2b9-af9f0091a6ed None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:25.903779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa33a8-436b-4dca-a2b9-af9f0091a6ed None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:25.910273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 348/694] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:25.910273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa33a8-436b-4dca-a2b9-af9f0091a6ed None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:26.000675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:26.012346 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 347/695] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:53:26.015134 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a65537-ca37-43a3-b2cd-fc63dea2bca8 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:26.015682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a65537-ca37-43a3-b2cd-fc63dea2bca8 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:26.016120 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a65537-ca37-43a3-b2cd-fc63dea2bca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:26.039327 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-54a65537-ca37-43a3-b2cd-fc63dea2bca8 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:26.041870 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-54a65537-ca37-43a3-b2cd-fc63dea2bca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:26.050597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa33a8-436b-4dca-a2b9-af9f0091a6ed None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:26.053285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:26.091922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d58f8-5b37-4144-a36e-46fa28182299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:26.092743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d58f8-5b37-4144-a36e-46fa28182299 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:26.093029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d58f8-5b37-4144-a36e-46fa28182299 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:26.093306 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d58f8-5b37-4144-a36e-46fa28182299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:26.106082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d58f8-5b37-4144-a36e-46fa28182299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:26.108534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d58f8-5b37-4144-a36e-46fa28182299 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:26.124062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 348/696] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:26.125168 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a65537-ca37-43a3-b2cd-fc63dea2bca8 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:26.128584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-db3d58f8-5b37-4144-a36e-46fa28182299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:26.128584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:26.128584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:26.128584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:26.128584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:26.128584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:26.128584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:26.128584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:26.128584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:26.128584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:26.128584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:26.128584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:26.128584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:26.128584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:26.128584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:26.129513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:26.130427 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:26.130427 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:26.130427 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:26.130427 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:26.135700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 349/697] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:26.136629 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d58f8-5b37-4144-a36e-46fa28182299 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:26.261146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d58f8-5b37-4144-a36e-46fa28182299 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:26.263778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a65537-ca37-43a3-b2cd-fc63dea2bca8 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:26.294899 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:26.302221 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938f06c-a401-4a94-aeb7-da95fc34b76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:26.303200 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938f06c-a401-4a94-aeb7-da95fc34b76a None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:26.303492 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938f06c-a401-4a94-aeb7-da95fc34b76a None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:26.303770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938f06c-a401-4a94-aeb7-da95fc34b76a None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:26.304506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938f06c-a401-4a94-aeb7-da95fc34b76a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:26.304814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938f06c-a401-4a94-aeb7-da95fc34b76a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:26.305069 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121c442-bee3-4874-b1f5-46a8eb41e968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:26.307000 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121c442-bee3-4874-b1f5-46a8eb41e968 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:26.312203 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938f06c-a401-4a94-aeb7-da95fc34b76a None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:26.312347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121c442-bee3-4874-b1f5-46a8eb41e968 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:26.312347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121c442-bee3-4874-b1f5-46a8eb41e968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:26.319691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121c442-bee3-4874-b1f5-46a8eb41e968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:26.321744 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121c442-bee3-4874-b1f5-46a8eb41e968 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:26.326047 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 350/698] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:26.327225 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938f06c-a401-4a94-aeb7-da95fc34b76a None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:26.343290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-3121c442-bee3-4874-b1f5-46a8eb41e968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:26.343290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:26.343290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:26.343290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:26.343290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:26.343290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:26.343290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:26.343290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:26.343290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:26.343290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:26.343290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:26.343290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:26.343290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:26.343290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:26.343290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:26.344475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:26.345573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:26.345573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:26.345573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:26.345573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:26.351922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:26.353403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 349/699] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:26.355014 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121c442-bee3-4874-b1f5-46a8eb41e968 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:26.440241 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938f06c-a401-4a94-aeb7-da95fc34b76a None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:26.467629 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121c442-bee3-4874-b1f5-46a8eb41e968 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:26.479566 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd6237-cb2b-4483-9b84-142d5e4a25f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:26.480382 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd6237-cb2b-4483-9b84-142d5e4a25f3 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:26.480668 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd6237-cb2b-4483-9b84-142d5e4a25f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:26.480939 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd6237-cb2b-4483-9b84-142d5e4a25f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:26.491426 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd6237-cb2b-4483-9b84-142d5e4a25f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:26.493487 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd6237-cb2b-4483-9b84-142d5e4a25f3 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:26.504581 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874730a-72d0-4058-aa6f-b00aef2f4589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:26.505410 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874730a-72d0-4058-aa6f-b00aef2f4589 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:26.505724 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874730a-72d0-4058-aa6f-b00aef2f4589 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:26.506031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874730a-72d0-4058-aa6f-b00aef2f4589 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:26.513429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-c0cd6237-cb2b-4483-9b84-142d5e4a25f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:26.513429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:26.513429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:26.513429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:26.513429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:26.513429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:26.513429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:26.513429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:26.513429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:26.513429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:26.513429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:26.513429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:26.513429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:26.513429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:26.513429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:26.515051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:26.516188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:26.516188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:26.516188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:26.516188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:26.516188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 351/700] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:26.516188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd6237-cb2b-4483-9b84-142d5e4a25f3 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:26.520315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874730a-72d0-4058-aa6f-b00aef2f4589 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:26.521817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874730a-72d0-4058-aa6f-b00aef2f4589 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:26.537510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-5874730a-72d0-4058-aa6f-b00aef2f4589 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:26.537510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:26.537510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:26.537510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:26.537510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:26.537510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:26.537510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:26.537510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:26.537510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:26.537510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:26.537510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:26.537510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:26.537510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:26.537510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:26.537510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:26.538666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:26.539770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:26.539770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:26.539770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:26.539770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:26.539770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:26.539770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:26.539770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:26.539770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:26.543621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 350/701] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:26.545004 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a6864-21ba-4483-b133-58e649f01d65 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:26.545297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a6864-21ba-4483-b133-58e649f01d65 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:26.545581 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a6864-21ba-4483-b133-58e649f01d65 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:26.546939 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 351/702] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:26.547634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a6864-21ba-4483-b133-58e649f01d65 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:26.596702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:26.603410 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:26.640487 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd6237-cb2b-4483-9b84-142d5e4a25f3 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:26.669483 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a6864-21ba-4483-b133-58e649f01d65 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:26.676868 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fc8c7-b091-43f6-9889-a0bdd9c06611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:26.677537 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fc8c7-b091-43f6-9889-a0bdd9c06611 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:26.677823 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fc8c7-b091-43f6-9889-a0bdd9c06611 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:26.678102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fc8c7-b091-43f6-9889-a0bdd9c06611 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:26.678869 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fc8c7-b091-43f6-9889-a0bdd9c06611 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:26.679178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fc8c7-b091-43f6-9889-a0bdd9c06611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:26.681390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fc8c7-b091-43f6-9889-a0bdd9c06611 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:26.694438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 352/703] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:26.695166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fc8c7-b091-43f6-9889-a0bdd9c06611 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:26.717190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc9842-3dc7-4f3f-be80-9238d076b784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:26.717871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc9842-3dc7-4f3f-be80-9238d076b784 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:26.718162 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc9842-3dc7-4f3f-be80-9238d076b784 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:26.718444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc9842-3dc7-4f3f-be80-9238d076b784 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:26.719226 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc9842-3dc7-4f3f-be80-9238d076b784 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:26.719536 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc9842-3dc7-4f3f-be80-9238d076b784 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:26.720957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc9842-3dc7-4f3f-be80-9238d076b784 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:26.738394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 352/704] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:26.738394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83060464-dc02-4542-9041-6b71e9699118 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:26.738394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83060464-dc02-4542-9041-6b71e9699118 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:26.738394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83060464-dc02-4542-9041-6b71e9699118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:26.753870 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-83060464-dc02-4542-9041-6b71e9699118 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:26.755114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-83060464-dc02-4542-9041-6b71e9699118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:26.820001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fc8c7-b091-43f6-9889-a0bdd9c06611 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:26.842577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 353/705] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:26.842577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83060464-dc02-4542-9041-6b71e9699118 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:26.857811 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:26.867964 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3b602-4232-4d03-ae88-c0fbe408c25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:26.875702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3b602-4232-4d03-ae88-c0fbe408c25f None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:26.875702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3b602-4232-4d03-ae88-c0fbe408c25f None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:26.875702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3b602-4232-4d03-ae88-c0fbe408c25f None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:26.875702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3b602-4232-4d03-ae88-c0fbe408c25f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:26.875702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3b602-4232-4d03-ae88-c0fbe408c25f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:26.875702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3b602-4232-4d03-ae88-c0fbe408c25f None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:26.879944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:26.890882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 353/706] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:26.892117 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3b602-4232-4d03-ae88-c0fbe408c25f None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:26.965427 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83060464-dc02-4542-9041-6b71e9699118 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:27.009209 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88a7da-9784-4b25-9f0a-dcd8a83df95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:27.010263 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88a7da-9784-4b25-9f0a-dcd8a83df95d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:27.010685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88a7da-9784-4b25-9f0a-dcd8a83df95d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:27.011018 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88a7da-9784-4b25-9f0a-dcd8a83df95d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:27.013981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3b602-4232-4d03-ae88-c0fbe408c25f None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:27.019972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88a7da-9784-4b25-9f0a-dcd8a83df95d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:27.021769 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88a7da-9784-4b25-9f0a-dcd8a83df95d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:27.038975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-ae88a7da-9784-4b25-9f0a-dcd8a83df95d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:27.038975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:27.038975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:27.038975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:27.038975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:27.038975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:27.038975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:27.038975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:27.038975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:27.038975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:27.038975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:27.038975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:27.038975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:27.038975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:27.038975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:27.040295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:27.041274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:27.041274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:27.041274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:27.041274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:27.041274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:27.041274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:27.041274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:27.041274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:27.041274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 354/707] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:27.041775 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88a7da-9784-4b25-9f0a-dcd8a83df95d None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:27.059837 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f55bf-09b2-47ae-b4e0-ba560a39fc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:27.061037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f55bf-09b2-47ae-b4e0-ba560a39fc6d None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:27.061037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f55bf-09b2-47ae-b4e0-ba560a39fc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:27.061037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f55bf-09b2-47ae-b4e0-ba560a39fc6d None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:27.061789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f55bf-09b2-47ae-b4e0-ba560a39fc6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:27.063336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f55bf-09b2-47ae-b4e0-ba560a39fc6d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:27.141610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:27.146051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 354/708] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:53:26 2022] POST /identity/v3/users => generated 254 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:53:27.147228 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f55bf-09b2-47ae-b4e0-ba560a39fc6d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:27.165951 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88a7da-9784-4b25-9f0a-dcd8a83df95d None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:27.205978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0adea-bfcb-4164-85fa-ec087aa5d051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:27.206863 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0adea-bfcb-4164-85fa-ec087aa5d051 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:27.207151 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0adea-bfcb-4164-85fa-ec087aa5d051 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:27.207426 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0adea-bfcb-4164-85fa-ec087aa5d051 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:27.208098 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0adea-bfcb-4164-85fa-ec087aa5d051 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:27.209566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0adea-bfcb-4164-85fa-ec087aa5d051 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:27.214708 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:27.226955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-72e0adea-bfcb-4164-85fa-ec087aa5d051 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:53:27.226955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:27.226955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:53:27.226955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:53:27.226955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:53:27.226955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:53:27.226955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:53:27.226955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:27.226955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:53:27.226955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:27.226955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:27.226955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:27.226955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:27.226955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:27.226955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:27.226955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:27.228169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:27.228169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:27.228169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:27.228169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:27.228169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:27.228169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:27.228169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:27.228169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:53:27.228169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:53:27.228169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:53:27.228169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:53:27.228169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:27.228169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:27.228169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:27.228169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:27.228169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:27.229089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:27.229089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:27.229089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:27.229089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:27.229089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:27.229089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:27.229089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:27.229089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:27.229089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:27.229089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:27.229089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:27.229089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:53:27.229089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:27.229089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:53:27.229089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:27.229089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 355/709] 10.222.0.83 () {56 vars in 1129 bytes} [Mon Jul 18 14:53:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:27.230674 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0adea-bfcb-4164-85fa-ec087aa5d051 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:27.285839 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f55bf-09b2-47ae-b4e0-ba560a39fc6d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:27.331443 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35614498-66fb-4150-bb35-be97ac8f9676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:27.332304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35614498-66fb-4150-bb35-be97ac8f9676 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:27.333125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35614498-66fb-4150-bb35-be97ac8f9676 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:27.333508 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35614498-66fb-4150-bb35-be97ac8f9676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:27.334876 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35614498-66fb-4150-bb35-be97ac8f9676 None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=7e1355d18fb049468651b3675c217d60) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:27.335204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35614498-66fb-4150-bb35-be97ac8f9676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:27.336811 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35614498-66fb-4150-bb35-be97ac8f9676 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:27.343849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 355/710] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:27 2022] GET /identity/v3/role_assignments?user.id=7e1355d18fb049468651b3675c217d60&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=5891920310b54787bd1152d736d2a3f1 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:27.345703 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f492f-751f-4e27-9c97-09de9d3e4799 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:27.345999 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f492f-751f-4e27-9c97-09de9d3e4799 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:27.346281 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f492f-751f-4e27-9c97-09de9d3e4799 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:27.347624 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 356/711] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:27.351482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f492f-751f-4e27-9c97-09de9d3e4799 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:27.368269 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0adea-bfcb-4164-85fa-ec087aa5d051 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:27.424734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b48f4-bd37-4ad9-b09a-3a95918ca019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:27.425217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b48f4-bd37-4ad9-b09a-3a95918ca019 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:27.425546 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b48f4-bd37-4ad9-b09a-3a95918ca019 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:27.425831 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b48f4-bd37-4ad9-b09a-3a95918ca019 None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:27.427617 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b48f4-bd37-4ad9-b09a-3a95918ca019 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:27.427617 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b48f4-bd37-4ad9-b09a-3a95918ca019 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:27.428627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b48f4-bd37-4ad9-b09a-3a95918ca019 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:27.448775 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 356/712] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:27.450699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032e49-7be0-4ec5-a388-f07920a0e61c None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:27.451213 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032e49-7be0-4ec5-a388-f07920a0e61c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:27.451705 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032e49-7be0-4ec5-a388-f07920a0e61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:27.472342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-32032e49-7be0-4ec5-a388-f07920a0e61c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:27.474133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-32032e49-7be0-4ec5-a388-f07920a0e61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:27.487600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:27.507841 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f492f-751f-4e27-9c97-09de9d3e4799 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:27.518838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:27.563554 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da4bbd-eae2-4392-bcdb-9ad83cebcf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:27.564540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da4bbd-eae2-4392-bcdb-9ad83cebcf71 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:27.564832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da4bbd-eae2-4392-bcdb-9ad83cebcf71 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:27.565115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da4bbd-eae2-4392-bcdb-9ad83cebcf71 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:27.565838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da4bbd-eae2-4392-bcdb-9ad83cebcf71 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:27.566149 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da4bbd-eae2-4392-bcdb-9ad83cebcf71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:27.567795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da4bbd-eae2-4392-bcdb-9ad83cebcf71 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:27.572054 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 357/713] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:27.573290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32032e49-7be0-4ec5-a388-f07920a0e61c None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:27.580697 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 357/714] 10.222.0.83 () {56 vars in 1124 bytes} [Mon Jul 18 14:53:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:27.586122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da4bbd-eae2-4392-bcdb-9ad83cebcf71 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:27.709450 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32032e49-7be0-4ec5-a388-f07920a0e61c None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:27.725410 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da4bbd-eae2-4392-bcdb-9ad83cebcf71 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:27.749882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813af7a-a61b-4730-936b-5ffde1c3d1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:27.750582 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813af7a-a61b-4730-936b-5ffde1c3d1dd None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:27.750888 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813af7a-a61b-4730-936b-5ffde1c3d1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:27.751192 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813af7a-a61b-4730-936b-5ffde1c3d1dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:27.752038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813af7a-a61b-4730-936b-5ffde1c3d1dd None admin] RBAC: Adding query filter params (role.id=29b8b950372848ffa9a75e778ee66b4c, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=f88dfe620ff04ffe88088a8a23c7d6e9) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:27.753000 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813af7a-a61b-4730-936b-5ffde1c3d1dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:27.754068 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813af7a-a61b-4730-936b-5ffde1c3d1dd None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:27.766650 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 358/715] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:27 2022] GET /identity/v3/role_assignments?user.id=f88dfe620ff04ffe88088a8a23c7d6e9&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=29b8b950372848ffa9a75e778ee66b4c => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:27.767996 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813af7a-a61b-4730-936b-5ffde1c3d1dd None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:27.774798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda17b8-db57-4f90-8ed1-130660e48a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:27.776637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda17b8-db57-4f90-8ed1-130660e48a81 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:27.777148 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda17b8-db57-4f90-8ed1-130660e48a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:27.777634 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda17b8-db57-4f90-8ed1-130660e48a81 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:27.778673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda17b8-db57-4f90-8ed1-130660e48a81 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:27.779257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda17b8-db57-4f90-8ed1-130660e48a81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:27.780942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda17b8-db57-4f90-8ed1-130660e48a81 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:27.788219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 358/716] 10.222.0.83 () {56 vars in 1124 bytes} [Mon Jul 18 14:53:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:27.790367 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:27.950489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813af7a-a61b-4730-936b-5ffde1c3d1dd None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:28.011196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3692d7-18eb-45b5-9f8d-2551b7ff4d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:28.014382 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3692d7-18eb-45b5-9f8d-2551b7ff4d85 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:28.014382 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3692d7-18eb-45b5-9f8d-2551b7ff4d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:28.014382 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3692d7-18eb-45b5-9f8d-2551b7ff4d85 None admin] PATH_INFO: `/v3/services/network` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:28.014382 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3692d7-18eb-45b5-9f8d-2551b7ff4d85 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:28.014786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3692d7-18eb-45b5-9f8d-2551b7ff4d85 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-5a3692d7-18eb-45b5-9f8d-2551b7ff4d85 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:28.030378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:28.031497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:28.031497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:28.031497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:28.031497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:28.031497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:28.031497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:28.031497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:53:28.031497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:53:28.031497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:53:28.031497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:53:28.031497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:28.031497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:28.031497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:28.031497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:28.031497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:28.031497 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:28.032561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:28.032561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:28.032561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:28.032561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:28.032561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:28.032561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:28.032561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:28.032561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:28.032561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:28.032561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:28.032561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:53:28.032561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:28.032561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:53:28.032561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:28.032561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 359/717] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:27 2022] GET /identity/v3/services/network => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:28.039504 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda17b8-db57-4f90-8ed1-130660e48a81 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:28.089488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:28.171563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda17b8-db57-4f90-8ed1-130660e48a81 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:28.214086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f639f3-0318-490c-98e6-a763a39c91a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:28.215226 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f639f3-0318-490c-98e6-a763a39c91a7 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:28.215709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f639f3-0318-490c-98e6-a763a39c91a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:28.216202 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f639f3-0318-490c-98e6-a763a39c91a7 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:28.217140 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f639f3-0318-490c-98e6-a763a39c91a7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:28.217653 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f639f3-0318-490c-98e6-a763a39c91a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:28.219446 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f639f3-0318-490c-98e6-a763a39c91a7 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:28.225704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 359/718] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Jul 18 14:53:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:28.234363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3692d7-18eb-45b5-9f8d-2551b7ff4d85 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:28.346615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:28.388830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3692d7-18eb-45b5-9f8d-2551b7ff4d85 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:28.437146 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40363ce7-63db-441f-bea4-707adcd9f909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:28.437839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40363ce7-63db-441f-bea4-707adcd9f909 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:28.438126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40363ce7-63db-441f-bea4-707adcd9f909 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:28.438410 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40363ce7-63db-441f-bea4-707adcd9f909 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:28.439201 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40363ce7-63db-441f-bea4-707adcd9f909 None admin] RBAC: Adding query filter params (type=network) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:28.439608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40363ce7-63db-441f-bea4-707adcd9f909 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:28.441106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40363ce7-63db-441f-bea4-707adcd9f909 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:28.452744 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 360/719] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Jul 18 14:53:28 2022] GET /identity/v3/services?type=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:29.718472 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e274-a984-479e-ac64-357a074bb9f9 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:29.719044 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e274-a984-479e-ac64-357a074bb9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:29.719044 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e274-a984-479e-ac64-357a074bb9f9 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:29.720859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 360/720] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:29.725793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b076b-0fdc-42db-97bc-c63b159c8728 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:29.726258 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b076b-0fdc-42db-97bc-c63b159c8728 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:29.726350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b076b-0fdc-42db-97bc-c63b159c8728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:29.749561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-2a3b076b-0fdc-42db-97bc-c63b159c8728 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:29.750809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-2a3b076b-0fdc-42db-97bc-c63b159c8728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:29.851106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 361/721] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:29.900300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0e274-a984-479e-ac64-357a074bb9f9 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:30.052926 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:30.060050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0e274-a984-479e-ac64-357a074bb9f9 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:30.103426 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27178907-e07a-47ed-b0bd-439ec4b77ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:30.104262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27178907-e07a-47ed-b0bd-439ec4b77ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:30.104595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27178907-e07a-47ed-b0bd-439ec4b77ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:30.104870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27178907-e07a-47ed-b0bd-439ec4b77ef4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:30.122143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27178907-e07a-47ed-b0bd-439ec4b77ef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:30.130069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-27178907-e07a-47ed-b0bd-439ec4b77ef4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:30.130069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:30.130069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:30.130069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:30.130069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:30.130069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:30.130069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:30.130069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:30.130069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:30.130069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:30.130069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:30.130069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:30.130069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:30.130069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:30.130069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:30.130069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:30.131088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:30.131088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:30.131088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:30.131088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:30.131088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:30.131088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:30.131088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:30.131088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:30.131088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:30.131088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:30.131088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:30.131088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:30.131088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:30.131088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:30.131088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:30.131088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:30.131940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:30.131940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:30.131940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:30.131940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:30.131940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:30.131940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:30.131940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:30.131940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:30.131940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:30.131940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:30.131940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:30.131940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:30.131940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 361/722] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:30.137672 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27178907-e07a-47ed-b0bd-439ec4b77ef4 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:30.137861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d110bc-35e8-4d56-8b36-0845f29627b9 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:30.138164 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d110bc-35e8-4d56-8b36-0845f29627b9 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:30.138351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d110bc-35e8-4d56-8b36-0845f29627b9 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:30.141138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 362/723] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:30.145585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ebd3b-5cd7-4e4e-af20-41cb8239c580 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:30.145873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ebd3b-5cd7-4e4e-af20-41cb8239c580 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:30.146161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ebd3b-5cd7-4e4e-af20-41cb8239c580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:30.163704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-240ebd3b-5cd7-4e4e-af20-41cb8239c580 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:30.165537 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-240ebd3b-5cd7-4e4e-af20-41cb8239c580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:30.235002 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:30.241118 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 363/724] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:53:30.264835 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ebd3b-5cd7-4e4e-af20-41cb8239c580 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:30.327812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27178907-e07a-47ed-b0bd-439ec4b77ef4 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:30.412636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643b877-11cd-499c-aab8-36ae540ecc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:30.413328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643b877-11cd-499c-aab8-36ae540ecc53 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:30.413618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643b877-11cd-499c-aab8-36ae540ecc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:30.413954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643b877-11cd-499c-aab8-36ae540ecc53 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:30.419179 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643b877-11cd-499c-aab8-36ae540ecc53 None admin] RBAC: Adding query filter params (name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:30.419179 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643b877-11cd-499c-aab8-36ae540ecc53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:30.420527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643b877-11cd-499c-aab8-36ae540ecc53 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:30.444939 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 362/725] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:30.449085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643b877-11cd-499c-aab8-36ae540ecc53 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:30.481757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ebd3b-5cd7-4e4e-af20-41cb8239c580 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:30.498476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:30.528374 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721ce3f-638a-4a29-8436-41f3ba6a3743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:30.529203 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721ce3f-638a-4a29-8436-41f3ba6a3743 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:30.529493 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721ce3f-638a-4a29-8436-41f3ba6a3743 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:30.529765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721ce3f-638a-4a29-8436-41f3ba6a3743 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:30.554640 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721ce3f-638a-4a29-8436-41f3ba6a3743 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:30.567117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-e721ce3f-638a-4a29-8436-41f3ba6a3743 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:53:30.567117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:30.567117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:30.567117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:30.567117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:30.567117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:30.567117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:30.567117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:30.567117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:30.567117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:30.567117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:30.567117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:30.567117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:30.567117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:30.567117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:30.567117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:30.568150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:30.568150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:30.568150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:30.568150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:30.568150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:30.568150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:30.568150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:30.568150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:30.568150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:30.568150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:30.568150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:30.568150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:30.568150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:30.568150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:30.568150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:30.568150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:30.569005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:30.569005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:30.569005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:30.569005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:30.569005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:30.569005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:30.569005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:30.569005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:30.569005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:30.569005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:30.569005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:53:30.569005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:30.572000 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 364/726] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:53:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:30.573462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef22c92-900a-4bce-b512-871259335d2c None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:30.573752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef22c92-900a-4bce-b512-871259335d2c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:30.574056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef22c92-900a-4bce-b512-871259335d2c None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:30.575918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 365/727] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:30.581936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef22c92-900a-4bce-b512-871259335d2c None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:30.632080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643b877-11cd-499c-aab8-36ae540ecc53 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:30.655515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:30.672484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79872b-7b59-459c-ba6c-5a89c111dd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:30.673294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79872b-7b59-459c-ba6c-5a89c111dd03 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:30.673596 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79872b-7b59-459c-ba6c-5a89c111dd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:30.673945 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79872b-7b59-459c-ba6c-5a89c111dd03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:30.687876 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79872b-7b59-459c-ba6c-5a89c111dd03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:30.689420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79872b-7b59-459c-ba6c-5a89c111dd03 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:30.708550 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef22c92-900a-4bce-b512-871259335d2c None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:30.710106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-0d79872b-7b59-459c-ba6c-5a89c111dd03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:30.710106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:30.710106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:30.710106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:30.710106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:30.710106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:30.710106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:30.710106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:30.710106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:30.710106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:30.710106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:30.710106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:30.710106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:30.710106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:30.710106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:30.711131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:30.712682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:30.712682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:30.712682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:30.712682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:30.712682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 363/728] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:30.713078 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f4390-9714-4c15-b6de-127e378b5f20 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:30.713078 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f4390-9714-4c15-b6de-127e378b5f20 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:30.713690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f4390-9714-4c15-b6de-127e378b5f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:30.731538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-6a4f4390-9714-4c15-b6de-127e378b5f20 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:30.733013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6a4f4390-9714-4c15-b6de-127e378b5f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:30.760345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cff82-65c4-457a-855a-b001334654b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:30.760345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cff82-65c4-457a-855a-b001334654b8 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:30.761079 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cff82-65c4-457a-855a-b001334654b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:30.761079 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cff82-65c4-457a-855a-b001334654b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:30.762593 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cff82-65c4-457a-855a-b001334654b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:30.763064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cff82-65c4-457a-855a-b001334654b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:30.765796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cff82-65c4-457a-855a-b001334654b8 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:30.782770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 366/729] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:53:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:30.782770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9672e6-2bc7-44bd-9e08-3387bebf57bc None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:30.782770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9672e6-2bc7-44bd-9e08-3387bebf57bc None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:30.782770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9672e6-2bc7-44bd-9e08-3387bebf57bc None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:30.783807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 367/730] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:30.784628 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9672e6-2bc7-44bd-9e08-3387bebf57bc None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:30.815075 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 364/731] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:30.817197 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f4390-9714-4c15-b6de-127e378b5f20 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:30.821365 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:30.931646 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9672e6-2bc7-44bd-9e08-3387bebf57bc None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:30.950587 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f4390-9714-4c15-b6de-127e378b5f20 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:30.963297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:30.992224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6992e3-2e35-4072-855d-07064cefbcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:30.994737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6992e3-2e35-4072-855d-07064cefbcda None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:30.995524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6992e3-2e35-4072-855d-07064cefbcda None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:30.996035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6992e3-2e35-4072-855d-07064cefbcda None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:30.997274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6992e3-2e35-4072-855d-07064cefbcda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:30.998177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6992e3-2e35-4072-855d-07064cefbcda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:30.999905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6992e3-2e35-4072-855d-07064cefbcda None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:31.002989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7149e-e0a9-43db-872d-c2ab349bfeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:31.003808 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7149e-e0a9-43db-872d-c2ab349bfeee None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:31.004098 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7149e-e0a9-43db-872d-c2ab349bfeee None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:31.004378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7149e-e0a9-43db-872d-c2ab349bfeee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:31.015465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7149e-e0a9-43db-872d-c2ab349bfeee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:31.017316 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 368/732] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:31.018903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e90497-0cf3-4c3b-a37f-212afbb75c09 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:31.019029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7149e-e0a9-43db-872d-c2ab349bfeee None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:31.019491 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e90497-0cf3-4c3b-a37f-212afbb75c09 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:31.019989 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e90497-0cf3-4c3b-a37f-212afbb75c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:31.040494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-28e90497-0cf3-4c3b-a37f-212afbb75c09 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:31.041830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-28e90497-0cf3-4c3b-a37f-212afbb75c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:31.042011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-39a7149e-e0a9-43db-872d-c2ab349bfeee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:31.042011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:31.042011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:31.042011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:31.042011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:31.042011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:31.042011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:31.042011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:31.042011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:31.042011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:31.042011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:31.042011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:31.042011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:31.042011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:31.042011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:31.043486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:31.045173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:31.045173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:31.045173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:31.045173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:31.047946 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 365/733] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:31.048863 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7149e-e0a9-43db-872d-c2ab349bfeee None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:31.121133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 369/734] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:31.121770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:31.121887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e90497-0cf3-4c3b-a37f-212afbb75c09 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:31.178603 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7149e-e0a9-43db-872d-c2ab349bfeee None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:31.221671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb4bc7-d2f0-4ed4-8ded-96a50ce90d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:31.223221 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb4bc7-d2f0-4ed4-8ded-96a50ce90d2a None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:31.223879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb4bc7-d2f0-4ed4-8ded-96a50ce90d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:31.224227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb4bc7-d2f0-4ed4-8ded-96a50ce90d2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:31.234930 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e90497-0cf3-4c3b-a37f-212afbb75c09 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:31.237955 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb4bc7-d2f0-4ed4-8ded-96a50ce90d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:31.246156 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:31.247118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-4bcb4bc7-d2f0-4ed4-8ded-96a50ce90d2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:53:31.247118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:31.247118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:31.247118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:31.247118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:31.247118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:31.247118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:31.247118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:31.247118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:31.247118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:31.247118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:31.247118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:31.247118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:31.247118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:31.247118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:31.247118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:31.248486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:31.248486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:31.248486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:31.248486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:31.248486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:31.248486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:31.248486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:31.248486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:31.248486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:31.248486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:31.248486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:31.248486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:31.248486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:31.248486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:31.248486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:31.248486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:31.249352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:31.249352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:31.249352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:31.249352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:31.249352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:31.249352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:31.249352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:31.249352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:31.249352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:31.249352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:31.249352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:53:31.249352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:31.249352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 366/735] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:53:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:31.249352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb4bc7-d2f0-4ed4-8ded-96a50ce90d2a None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:31.281106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc407f5-cb60-4013-9816-8508a1c50628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:31.281963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc407f5-cb60-4013-9816-8508a1c50628 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:31.282247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc407f5-cb60-4013-9816-8508a1c50628 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:31.282517 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc407f5-cb60-4013-9816-8508a1c50628 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:31.293412 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc407f5-cb60-4013-9816-8508a1c50628 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:31.294980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc407f5-cb60-4013-9816-8508a1c50628 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:31.311843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-bfc407f5-cb60-4013-9816-8508a1c50628 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:53:31.311843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:31.311843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:31.311843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:31.311843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:31.311843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:31.311843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:31.311843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:31.311843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:31.311843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:31.311843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:31.311843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:31.311843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:31.311843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:31.311843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:31.311843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:31.312918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:31.313877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:31.313877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:31.313877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:31.313877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:31.313877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:31.313877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:31.313877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:31.313877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:31.313877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:53:31.313877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:31.313877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 370/736] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Jul 18 14:53:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:31.313877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc407f5-cb60-4013-9816-8508a1c50628 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:31.380926 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb4bc7-d2f0-4ed4-8ded-96a50ce90d2a None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:31.399318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:31.419164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6ec33-fccd-4e9d-bdb8-5a6d0333bc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:31.419839 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6ec33-fccd-4e9d-bdb8-5a6d0333bc1b None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:31.420123 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6ec33-fccd-4e9d-bdb8-5a6d0333bc1b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:31.420412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6ec33-fccd-4e9d-bdb8-5a6d0333bc1b None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:31.421173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6ec33-fccd-4e9d-bdb8-5a6d0333bc1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:31.421487 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6ec33-fccd-4e9d-bdb8-5a6d0333bc1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:31.422910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6ec33-fccd-4e9d-bdb8-5a6d0333bc1b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:31.426836 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc407f5-cb60-4013-9816-8508a1c50628 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:31.435141 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 367/737] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:31.435922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6ec33-fccd-4e9d-bdb8-5a6d0333bc1b None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:31.467640 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2a3e2-7ef7-4097-a265-b2f0e174ca54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:31.468305 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a3e2-7ef7-4097-a265-b2f0e174ca54 None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:31.468621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a3e2-7ef7-4097-a265-b2f0e174ca54 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:31.468881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a3e2-7ef7-4097-a265-b2f0e174ca54 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:31.469606 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2a3e2-7ef7-4097-a265-b2f0e174ca54 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:31.471050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2a3e2-7ef7-4097-a265-b2f0e174ca54 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:31.539451 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6ec33-fccd-4e9d-bdb8-5a6d0333bc1b None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:31.567806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 371/738] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Jul 18 14:53:31 2022] POST /identity/v3/services => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:53:31.568747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2a3e2-7ef7-4097-a265-b2f0e174ca54 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:31.592939 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb563e-5ed1-45fb-8f8f-51bb933fea9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:31.593637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb563e-5ed1-45fb-8f8f-51bb933fea9d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:31.593928 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb563e-5ed1-45fb-8f8f-51bb933fea9d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:31.594211 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb563e-5ed1-45fb-8f8f-51bb933fea9d None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:31.595111 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb563e-5ed1-45fb-8f8f-51bb933fea9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:31.595426 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb563e-5ed1-45fb-8f8f-51bb933fea9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:31.597099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb563e-5ed1-45fb-8f8f-51bb933fea9d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:31.614437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 368/739] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:53:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:31.614437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1f6b6-cda5-41b8-a6f2-c2cf1f851bb0 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:31.614437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1f6b6-cda5-41b8-a6f2-c2cf1f851bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:31.614437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1f6b6-cda5-41b8-a6f2-c2cf1f851bb0 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:31.614437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 369/740] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:31.614437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1f6b6-cda5-41b8-a6f2-c2cf1f851bb0 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:31.630394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:31.659468 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:31.720825 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2a3e2-7ef7-4097-a265-b2f0e174ca54 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:31.741109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1f6b6-cda5-41b8-a6f2-c2cf1f851bb0 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:31.768984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79015e69-24c6-4680-b712-1ff0fc2b8e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:31.769777 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79015e69-24c6-4680-b712-1ff0fc2b8e3f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:31.769978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79015e69-24c6-4680-b712-1ff0fc2b8e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:31.770220 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79015e69-24c6-4680-b712-1ff0fc2b8e3f None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:31.771101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79015e69-24c6-4680-b712-1ff0fc2b8e3f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:31.771434 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79015e69-24c6-4680-b712-1ff0fc2b8e3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:31.773577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79015e69-24c6-4680-b712-1ff0fc2b8e3f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:31.788708 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b787ace-f248-40af-a8e9-d6047f673fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:31.789439 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b787ace-f248-40af-a8e9-d6047f673fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:31.789728 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b787ace-f248-40af-a8e9-d6047f673fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:31.790010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b787ace-f248-40af-a8e9-d6047f673fb7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:31.796415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 372/741] 10.222.0.83 () {56 vars in 1149 bytes} [Mon Jul 18 14:53:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:31.798698 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79015e69-24c6-4680-b712-1ff0fc2b8e3f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:31.799080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b787ace-f248-40af-a8e9-d6047f673fb7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:31.800864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b787ace-f248-40af-a8e9-d6047f673fb7 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:31.819618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-4b787ace-f248-40af-a8e9-d6047f673fb7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:53:31.819618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:31.819618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:31.819618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:31.819618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:31.819618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:31.819618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:31.819618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:31.819618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:31.819618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:31.819618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:31.819618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:31.819618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:31.819618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:31.819618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:31.819618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:31.821285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:31.823351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:31.823351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:31.823351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:31.823351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:31.823351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:31.823351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:31.823351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:31.823351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:31.823351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:53:31.823351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:31.823351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 370/742] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Jul 18 14:53:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:31.823351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6260c1-f9d0-43d0-9ab2-5efd6b2f1846 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:31.824054 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6260c1-f9d0-43d0-9ab2-5efd6b2f1846 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:31.824054 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6260c1-f9d0-43d0-9ab2-5efd6b2f1846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:31.844513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-cd6260c1-f9d0-43d0-9ab2-5efd6b2f1846 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:31.845759 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-cd6260c1-f9d0-43d0-9ab2-5efd6b2f1846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:31.906359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:31.921500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79015e69-24c6-4680-b712-1ff0fc2b8e3f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:31.930759 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 371/743] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:53:31.930759 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6260c1-f9d0-43d0-9ab2-5efd6b2f1846 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:31.961484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:31.975841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6293d-3450-4b59-b8fe-1411797ff51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:31.975841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6293d-3450-4b59-b8fe-1411797ff51c None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:31.975841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6293d-3450-4b59-b8fe-1411797ff51c None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:31.975841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6293d-3450-4b59-b8fe-1411797ff51c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:31.985479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6293d-3450-4b59-b8fe-1411797ff51c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:31.987033 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6293d-3450-4b59-b8fe-1411797ff51c None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:32.007351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-70f6293d-3450-4b59-b8fe-1411797ff51c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:32.007351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:32.007351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:32.007351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:32.007351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:32.007351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:32.007351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:32.007351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:32.007351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:32.007351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:32.007351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:32.007351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:32.007351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:32.007351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:32.007351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:32.009722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:32.013380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:32.013380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:32.013380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:32.013380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:32.013380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 373/744] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:32.013380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6293d-3450-4b59-b8fe-1411797ff51c None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:32.057863 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6260c1-f9d0-43d0-9ab2-5efd6b2f1846 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:32.097959 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52a13b-9b5f-4d83-aca0-763a6b12a081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:32.098656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52a13b-9b5f-4d83-aca0-763a6b12a081 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:32.098951 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52a13b-9b5f-4d83-aca0-763a6b12a081 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:32.099233 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52a13b-9b5f-4d83-aca0-763a6b12a081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:32.119622 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6293d-3450-4b59-b8fe-1411797ff51c None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:32.157231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a11b04-7a56-4e2d-9cac-5d885cca2d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:32.157917 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a11b04-7a56-4e2d-9cac-5d885cca2d14 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:32.158204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a11b04-7a56-4e2d-9cac-5d885cca2d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:32.158488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a11b04-7a56-4e2d-9cac-5d885cca2d14 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:32.159407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a11b04-7a56-4e2d-9cac-5d885cca2d14 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:32.159726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a11b04-7a56-4e2d-9cac-5d885cca2d14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:32.161241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a11b04-7a56-4e2d-9cac-5d885cca2d14 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:32.171130 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:32.183112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52a13b-9b5f-4d83-aca0-763a6b12a081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:32.187418 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52a13b-9b5f-4d83-aca0-763a6b12a081 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:32.187536 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 374/745] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Jul 18 14:53:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:32.187536 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a11b04-7a56-4e2d-9cac-5d885cca2d14 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:32.219141 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:32.297980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a11b04-7a56-4e2d-9cac-5d885cca2d14 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:32.305344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 372/746] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:31 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:53:32.306187 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52a13b-9b5f-4d83-aca0-763a6b12a081 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:32.357810 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeafe0e-0bd3-4327-8bad-dcad1490355f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:32.358569 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeafe0e-0bd3-4327-8bad-dcad1490355f None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:32.358864 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeafe0e-0bd3-4327-8bad-dcad1490355f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:32.359196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeafe0e-0bd3-4327-8bad-dcad1490355f None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:32.360044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeafe0e-0bd3-4327-8bad-dcad1490355f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:32.361867 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeafe0e-0bd3-4327-8bad-dcad1490355f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:32.428178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52a13b-9b5f-4d83-aca0-763a6b12a081 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:32.442983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 375/747] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Jul 18 14:53:32 2022] POST /identity/v3/services => generated 240 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:53:32.443915 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeafe0e-0bd3-4327-8bad-dcad1490355f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:32.467877 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:32.488115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20543bb1-3c9a-4c35-a741-76d4ae447158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:32.489009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20543bb1-3c9a-4c35-a741-76d4ae447158 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:32.489207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20543bb1-3c9a-4c35-a741-76d4ae447158 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:32.489462 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20543bb1-3c9a-4c35-a741-76d4ae447158 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:32.490277 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20543bb1-3c9a-4c35-a741-76d4ae447158 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:32.490681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20543bb1-3c9a-4c35-a741-76d4ae447158 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:32.492318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20543bb1-3c9a-4c35-a741-76d4ae447158 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:32.513114 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 373/748] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:32.513114 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20543bb1-3c9a-4c35-a741-76d4ae447158 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:32.542393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:32.609595 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeafe0e-0bd3-4327-8bad-dcad1490355f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:32.649452 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20543bb1-3c9a-4c35-a741-76d4ae447158 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:32.654521 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba596140-09b3-4a1a-9d90-f16138b22045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:32.655369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba596140-09b3-4a1a-9d90-f16138b22045 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:32.655658 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba596140-09b3-4a1a-9d90-f16138b22045 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:32.655937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba596140-09b3-4a1a-9d90-f16138b22045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:32.667443 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba596140-09b3-4a1a-9d90-f16138b22045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:32.668952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba596140-09b3-4a1a-9d90-f16138b22045 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:32.692788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fa5d0-7c1f-44bb-8edc-ee2fab3d6dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:32.693049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-ba596140-09b3-4a1a-9d90-f16138b22045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:32.693049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:32.693049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:32.693049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:32.693049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:32.693049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:32.693049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:32.693049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:32.693049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:32.693049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:32.693049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:32.693049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:32.693049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:32.693049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:32.693049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:32.693898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:32.695088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:32.695088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:32.695088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:32.695088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:32.695088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 376/749] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:32.696262 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba596140-09b3-4a1a-9d90-f16138b22045 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:32.697480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fa5d0-7c1f-44bb-8edc-ee2fab3d6dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:32.698027 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fa5d0-7c1f-44bb-8edc-ee2fab3d6dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:32.698493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fa5d0-7c1f-44bb-8edc-ee2fab3d6dc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:32.712061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fa5d0-7c1f-44bb-8edc-ee2fab3d6dc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:32.714465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fa5d0-7c1f-44bb-8edc-ee2fab3d6dc1 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:32.738911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-985fa5d0-7c1f-44bb-8edc-ee2fab3d6dc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:32.738911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:32.738911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:32.738911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:32.738911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:32.738911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:32.738911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:32.738911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:32.738911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:32.738911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:32.738911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:32.738911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:32.738911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:32.738911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:32.738911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:32.739915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:32.740855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:32.740855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:32.740855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:32.740855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:32.744828 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 374/750] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:32.746663 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fa5d0-7c1f-44bb-8edc-ee2fab3d6dc1 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:32.796179 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:32.838751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba596140-09b3-4a1a-9d90-f16138b22045 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:32.850949 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:32.878519 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebc10c-13ff-4a79-a1af-cfa2a51ebda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:32.879549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebc10c-13ff-4a79-a1af-cfa2a51ebda5 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:32.879890 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebc10c-13ff-4a79-a1af-cfa2a51ebda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:32.880433 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebc10c-13ff-4a79-a1af-cfa2a51ebda5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:32.884676 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fa5d0-7c1f-44bb-8edc-ee2fab3d6dc1 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:32.891022 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebc10c-13ff-4a79-a1af-cfa2a51ebda5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:32.892760 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebc10c-13ff-4a79-a1af-cfa2a51ebda5 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:32.909057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-70ebc10c-13ff-4a79-a1af-cfa2a51ebda5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:53:32.909057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:32.909057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:32.909057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:32.909057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:32.909057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:32.909057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:32.909057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:32.909057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:32.909057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:32.909057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:32.909057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:32.909057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:32.909057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:32.909057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:32.909057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:32.910250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:32.911302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:32.911302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:32.911302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:32.911302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:32.911302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:32.911302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:32.911302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:32.911302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:32.911302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:53:32.911302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:32.912185 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 377/751] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:53:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:32.913038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebc10c-13ff-4a79-a1af-cfa2a51ebda5 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:32.931506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684af53-2c45-47d9-93af-a63cc096df8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:32.932336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684af53-2c45-47d9-93af-a63cc096df8b None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:32.932728 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684af53-2c45-47d9-93af-a63cc096df8b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:32.933114 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684af53-2c45-47d9-93af-a63cc096df8b None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:32.933960 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684af53-2c45-47d9-93af-a63cc096df8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:32.934481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684af53-2c45-47d9-93af-a63cc096df8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:32.936071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684af53-2c45-47d9-93af-a63cc096df8b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:32.953235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 375/752] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:32.953235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684af53-2c45-47d9-93af-a63cc096df8b None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:33.035266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebc10c-13ff-4a79-a1af-cfa2a51ebda5 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:33.056910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:33.067050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684af53-2c45-47d9-93af-a63cc096df8b None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:33.077386 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24de3b4-4a5b-4c15-bfc9-0afea5b4fcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:33.078318 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24de3b4-4a5b-4c15-bfc9-0afea5b4fcf3 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:33.078812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24de3b4-4a5b-4c15-bfc9-0afea5b4fcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:33.079239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24de3b4-4a5b-4c15-bfc9-0afea5b4fcf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:33.080256 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24de3b4-4a5b-4c15-bfc9-0afea5b4fcf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:33.080923 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24de3b4-4a5b-4c15-bfc9-0afea5b4fcf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:33.082928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24de3b4-4a5b-4c15-bfc9-0afea5b4fcf3 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:33.097659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 378/753] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:33.098963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24de3b4-4a5b-4c15-bfc9-0afea5b4fcf3 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:33.110257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06125383-7685-48a7-bb63-ffd75175d9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:33.111063 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06125383-7685-48a7-bb63-ffd75175d9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:33.111427 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06125383-7685-48a7-bb63-ffd75175d9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:33.111783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06125383-7685-48a7-bb63-ffd75175d9d0 None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:33.112768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06125383-7685-48a7-bb63-ffd75175d9d0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:33.113150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06125383-7685-48a7-bb63-ffd75175d9d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:33.114704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06125383-7685-48a7-bb63-ffd75175d9d0 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:33.134168 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:33.136170 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 376/754] 10.222.0.83 () {56 vars in 1135 bytes} [Mon Jul 18 14:53:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:33.137307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06125383-7685-48a7-bb63-ffd75175d9d0 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:33.212576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24de3b4-4a5b-4c15-bfc9-0afea5b4fcf3 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:33.250254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764984b-5ce1-4e5b-9b5d-ad8fcadd74b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:33.251453 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764984b-5ce1-4e5b-9b5d-ad8fcadd74b1 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:33.251831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06125383-7685-48a7-bb63-ffd75175d9d0 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:33.252097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764984b-5ce1-4e5b-9b5d-ad8fcadd74b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:33.254691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764984b-5ce1-4e5b-9b5d-ad8fcadd74b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:33.261741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764984b-5ce1-4e5b-9b5d-ad8fcadd74b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:33.263817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764984b-5ce1-4e5b-9b5d-ad8fcadd74b1 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:33.281441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-4764984b-5ce1-4e5b-9b5d-ad8fcadd74b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:33.281441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:33.281441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:33.281441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:33.281441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:33.281441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:33.281441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:33.281441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:33.281441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:33.281441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:33.281441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:33.281441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:33.281441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:33.281441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:33.281441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:33.282470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:33.283534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:33.283534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:33.283534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:33.283534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:33.283534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:33.283534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:33.283534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:33.283534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:33.293672 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30ba16-992f-4ae5-a280-1afb9f048dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:33.293985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 379/755] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:33.294734 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30ba16-992f-4ae5-a280-1afb9f048dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:33.295126 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30ba16-992f-4ae5-a280-1afb9f048dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:33.295506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30ba16-992f-4ae5-a280-1afb9f048dd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:33.295690 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764984b-5ce1-4e5b-9b5d-ad8fcadd74b1 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:33.305327 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30ba16-992f-4ae5-a280-1afb9f048dd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:33.305709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30ba16-992f-4ae5-a280-1afb9f048dd3 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:33.321070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-3b30ba16-992f-4ae5-a280-1afb9f048dd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:33.321070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:33.321070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:33.321070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:33.321070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:33.321070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:33.321070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:33.321070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:33.321070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:33.321070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:33.321070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:33.321070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:33.321070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:33.321070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:33.321070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:33.322109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:33.323086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:33.323086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:33.323086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:33.323086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:33.323086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:33.323086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:33.323086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:33.323086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:33.323086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 377/756] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:33.323605 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30ba16-992f-4ae5-a280-1afb9f048dd3 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:33.342758 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:33.438108 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:33.453063 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30ba16-992f-4ae5-a280-1afb9f048dd3 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:33.457777 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764984b-5ce1-4e5b-9b5d-ad8fcadd74b1 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:33.498489 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a3eea-01c2-481c-841f-6ea16e93c93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:33.499942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a3eea-01c2-481c-841f-6ea16e93c93d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:33.500892 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a3eea-01c2-481c-841f-6ea16e93c93d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:33.501682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a3eea-01c2-481c-841f-6ea16e93c93d None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:33.502855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a3eea-01c2-481c-841f-6ea16e93c93d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:33.503388 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a3eea-01c2-481c-841f-6ea16e93c93d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:33.505282 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a3eea-01c2-481c-841f-6ea16e93c93d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:33.505786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c804500-1e02-4682-8062-5edf0ca83598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:33.506664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c804500-1e02-4682-8062-5edf0ca83598 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:33.507049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c804500-1e02-4682-8062-5edf0ca83598 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:33.507452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c804500-1e02-4682-8062-5edf0ca83598 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:33.522106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 378/757] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:33.523310 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a3eea-01c2-481c-841f-6ea16e93c93d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:33.595071 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c804500-1e02-4682-8062-5edf0ca83598 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:33.596719 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c804500-1e02-4682-8062-5edf0ca83598 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:33.625598 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:33.663971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a3eea-01c2-481c-841f-6ea16e93c93d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:33.708091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae55a8-572f-47f8-aae7-c77078690c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:33.708628 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae55a8-572f-47f8-aae7-c77078690c23 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:33.708912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae55a8-572f-47f8-aae7-c77078690c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:33.709225 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae55a8-572f-47f8-aae7-c77078690c23 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:33.709970 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae55a8-572f-47f8-aae7-c77078690c23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:33.710283 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae55a8-572f-47f8-aae7-c77078690c23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:33.712235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae55a8-572f-47f8-aae7-c77078690c23 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:33.713125 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:33.728445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 380/758] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:53:33.732534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c804500-1e02-4682-8062-5edf0ca83598 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:33.733738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 379/759] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:33.735806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea99362-6077-4d0f-b611-7426477ca849 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:33.736114 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea99362-6077-4d0f-b611-7426477ca849 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:33.736376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea99362-6077-4d0f-b611-7426477ca849 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:33.738611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 380/760] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:33.739405 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea99362-6077-4d0f-b611-7426477ca849 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:33.857614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea99362-6077-4d0f-b611-7426477ca849 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:33.861123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c804500-1e02-4682-8062-5edf0ca83598 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:33.894829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:33.901404 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a5e63-a0a9-4d55-a918-154a3a231cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:33.903670 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a5e63-a0a9-4d55-a918-154a3a231cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:33.903670 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a5e63-a0a9-4d55-a918-154a3a231cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:33.903670 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a5e63-a0a9-4d55-a918-154a3a231cf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:33.903670 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a5e63-a0a9-4d55-a918-154a3a231cf0 None admin] RBAC: Adding query filter params (role.id=29b8b950372848ffa9a75e778ee66b4c, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=692fef632836474da09f0e8a009ffa4e) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:33.904019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a5e63-a0a9-4d55-a918-154a3a231cf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:33.905123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb5011-74ad-421a-ae79-03bec5278401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:33.905271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a5e63-a0a9-4d55-a918-154a3a231cf0 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:33.906045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb5011-74ad-421a-ae79-03bec5278401 None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:33.906328 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb5011-74ad-421a-ae79-03bec5278401 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:33.906623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb5011-74ad-421a-ae79-03bec5278401 None admin] PATH_INFO: `/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/7e1355d18fb049468651b3675c217d60/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:33.908601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb5011-74ad-421a-ae79-03bec5278401 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f777cd6a0c744e594f8e29ab58cf8bd, user_id=7e1355d18fb049468651b3675c217d60, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:33.910069 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb5011-74ad-421a-ae79-03bec5278401 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:33.911459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 381/761] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:33 2022] GET /identity/v3/role_assignments?user.id=692fef632836474da09f0e8a009ffa4e&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=29b8b950372848ffa9a75e778ee66b4c => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:33.919833 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a5e63-a0a9-4d55-a918-154a3a231cf0 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:34.026907 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 381/762] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:53:33 2022] PUT /identity/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/7e1355d18fb049468651b3675c217d60/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:53:34.028941 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44a77c-c5a2-4747-999c-a122b5a3dbf2 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:34.029457 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44a77c-c5a2-4747-999c-a122b5a3dbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:34.029946 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44a77c-c5a2-4747-999c-a122b5a3dbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:34.063996 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a5e63-a0a9-4d55-a918-154a3a231cf0 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:34.072253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-ab44a77c-c5a2-4747-999c-a122b5a3dbf2 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:34.073860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ab44a77c-c5a2-4747-999c-a122b5a3dbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:34.111138 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c1d4a-c603-49a3-9d00-fdcaec42e639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:34.112704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c1d4a-c603-49a3-9d00-fdcaec42e639 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:34.113055 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c1d4a-c603-49a3-9d00-fdcaec42e639 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:34.113403 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c1d4a-c603-49a3-9d00-fdcaec42e639 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:34.130838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c1d4a-c603-49a3-9d00-fdcaec42e639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:34.132693 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c1d4a-c603-49a3-9d00-fdcaec42e639 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:34.165034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-d60c1d4a-c603-49a3-9d00-fdcaec42e639 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:34.165034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:34.165034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:34.165034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:34.165034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:34.165034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:34.165034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:34.165034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:34.165034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:34.165034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:34.165034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:34.165034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:34.165034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:34.165034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:34.165034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:34.166162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:34.167121 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:34.167121 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:34.167121 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:34.167121 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:34.167121 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 382/763] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:34.173513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c1d4a-c603-49a3-9d00-fdcaec42e639 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:34.213635 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:34.267043 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 382/764] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:34.286758 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:34.303579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44a77c-c5a2-4747-999c-a122b5a3dbf2 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:34.350473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c1d4a-c603-49a3-9d00-fdcaec42e639 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:34.391992 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cd635-b00d-4b19-b240-7dfbfaaaf7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:34.393069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cd635-b00d-4b19-b240-7dfbfaaaf7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:34.393364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cd635-b00d-4b19-b240-7dfbfaaaf7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:34.393803 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cd635-b00d-4b19-b240-7dfbfaaaf7b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:34.394931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5cd635-b00d-4b19-b240-7dfbfaaaf7b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:34.395523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5cd635-b00d-4b19-b240-7dfbfaaaf7b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:34.397169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5cd635-b00d-4b19-b240-7dfbfaaaf7b6 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:34.418813 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 383/765] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:34.426081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cd635-b00d-4b19-b240-7dfbfaaaf7b6 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:34.460287 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44a77c-c5a2-4747-999c-a122b5a3dbf2 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:34.537202 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc2d1c-f395-4cac-aeee-ad05d143f2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:34.538407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc2d1c-f395-4cac-aeee-ad05d143f2ef None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:34.538959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc2d1c-f395-4cac-aeee-ad05d143f2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:34.539737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc2d1c-f395-4cac-aeee-ad05d143f2ef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:34.540625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc2d1c-f395-4cac-aeee-ad05d143f2ef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:34.544951 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc2d1c-f395-4cac-aeee-ad05d143f2ef None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:34.566679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-c4fc2d1c-f395-4cac-aeee-ad05d143f2ef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:53:34.566679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:34.566679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:53:34.566679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:53:34.566679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:53:34.566679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:53:34.566679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:53:34.566679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:34.566679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:53:34.566679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:34.566679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:34.566679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:34.566679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:34.566679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:34.566679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:34.566679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:34.567742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:34.567742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:34.567742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:34.567742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:34.567742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:34.567742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:34.567742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:34.567742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:53:34.567742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:53:34.567742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:53:34.567742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:53:34.567742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:34.567742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:34.567742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:34.567742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:34.567742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:34.568627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:34.568627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:34.568627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:34.568627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:34.568627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:34.568627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:34.568627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:34.568627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:34.568627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:34.568627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:34.568627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:34.568627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:53:34.568627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:34.568627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:53:34.568627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:34.572834 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 383/766] 10.222.0.83 () {56 vars in 1129 bytes} [Mon Jul 18 14:53:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:34.579343 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc2d1c-f395-4cac-aeee-ad05d143f2ef None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:34.618058 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cd635-b00d-4b19-b240-7dfbfaaaf7b6 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:34.628032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:34.667861 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40729de-8e6b-4dea-b986-ed0a07798e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:34.670847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40729de-8e6b-4dea-b986-ed0a07798e66 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:34.671666 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40729de-8e6b-4dea-b986-ed0a07798e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:34.673016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40729de-8e6b-4dea-b986-ed0a07798e66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:34.691672 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40729de-8e6b-4dea-b986-ed0a07798e66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:34.696849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40729de-8e6b-4dea-b986-ed0a07798e66 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:34.699381 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:34.722183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-e40729de-8e6b-4dea-b986-ed0a07798e66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:34.722183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:34.722183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:34.722183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:34.722183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:34.722183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:34.722183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:34.722183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:34.722183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:34.722183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:34.722183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:34.722183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:34.722183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:34.722183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:34.722183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:34.723262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:34.724208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:34.724208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:34.724208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:34.724208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:34.724208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:34.724208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:34.724208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:34.724208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:34.727068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 384/767] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:34.734589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40729de-8e6b-4dea-b986-ed0a07798e66 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:34.826508 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc2d1c-f395-4cac-aeee-ad05d143f2ef None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:34.884368 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8d7e5-2b95-4420-815b-85933f461016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:34.885060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8d7e5-2b95-4420-815b-85933f461016 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:34.885925 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8d7e5-2b95-4420-815b-85933f461016 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:34.886219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8d7e5-2b95-4420-815b-85933f461016 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:34.886978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8d7e5-2b95-4420-815b-85933f461016 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:34.887568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8d7e5-2b95-4420-815b-85933f461016 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:34.889268 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8d7e5-2b95-4420-815b-85933f461016 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:34.896644 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 384/768] 10.222.0.83 () {56 vars in 1124 bytes} [Mon Jul 18 14:53:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:34.899412 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2661b-d30d-4dd2-8d8e-b966e19d3e79 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:34.899903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2661b-d30d-4dd2-8d8e-b966e19d3e79 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:34.900328 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2661b-d30d-4dd2-8d8e-b966e19d3e79 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:34.900876 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40729de-8e6b-4dea-b986-ed0a07798e66 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:34.906147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 385/769] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:34.908160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2661b-d30d-4dd2-8d8e-b966e19d3e79 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:34.950677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4c628-a17c-48d4-8002-d9f796563d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:34.951483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4c628-a17c-48d4-8002-d9f796563d33 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:34.951771 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4c628-a17c-48d4-8002-d9f796563d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:34.952051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4c628-a17c-48d4-8002-d9f796563d33 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:34.952849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4c628-a17c-48d4-8002-d9f796563d33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:34.953159 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4c628-a17c-48d4-8002-d9f796563d33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:34.954619 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4c628-a17c-48d4-8002-d9f796563d33 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:34.969320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 385/770] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:34.970656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096cfda-85b8-4dd7-82b9-0e5539a287a5 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:34.970946 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096cfda-85b8-4dd7-82b9-0e5539a287a5 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:34.971220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096cfda-85b8-4dd7-82b9-0e5539a287a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:34.989812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-2096cfda-85b8-4dd7-82b9-0e5539a287a5 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:34.994234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2096cfda-85b8-4dd7-82b9-0e5539a287a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:35.017878 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:35.063366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2661b-d30d-4dd2-8d8e-b966e19d3e79 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:35.082929 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 386/771] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:35.084142 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096cfda-85b8-4dd7-82b9-0e5539a287a5 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:35.084866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:35.130676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e999a25-fd5f-4d26-9fc7-c0b2c0dd82c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:35.131543 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e999a25-fd5f-4d26-9fc7-c0b2c0dd82c9 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:35.131927 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e999a25-fd5f-4d26-9fc7-c0b2c0dd82c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:35.132436 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e999a25-fd5f-4d26-9fc7-c0b2c0dd82c9 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:35.134672 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e999a25-fd5f-4d26-9fc7-c0b2c0dd82c9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:35.135280 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e999a25-fd5f-4d26-9fc7-c0b2c0dd82c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:35.138197 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e999a25-fd5f-4d26-9fc7-c0b2c0dd82c9 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:35.147982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 386/772] 10.222.0.83 () {56 vars in 1124 bytes} [Mon Jul 18 14:53:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:35.149957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e999a25-fd5f-4d26-9fc7-c0b2c0dd82c9 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:35.218515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096cfda-85b8-4dd7-82b9-0e5539a287a5 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:35.264388 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5518278-cfe2-4f35-933b-3d38e2d23bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:35.265442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5518278-cfe2-4f35-933b-3d38e2d23bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:35.265909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5518278-cfe2-4f35-933b-3d38e2d23bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:35.266361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5518278-cfe2-4f35-933b-3d38e2d23bd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:35.267439 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5518278-cfe2-4f35-933b-3d38e2d23bd0 None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=f88dfe620ff04ffe88088a8a23c7d6e9) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:35.267912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5518278-cfe2-4f35-933b-3d38e2d23bd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:35.269682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5518278-cfe2-4f35-933b-3d38e2d23bd0 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:35.277537 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 387/773] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:35 2022] GET /identity/v3/role_assignments?user.id=f88dfe620ff04ffe88088a8a23c7d6e9&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=5891920310b54787bd1152d736d2a3f1 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:35.278741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5518278-cfe2-4f35-933b-3d38e2d23bd0 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:35.293675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e999a25-fd5f-4d26-9fc7-c0b2c0dd82c9 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:35.307895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:35.333282 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11e17e-6926-4170-b130-dfea2801346f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:35.334081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11e17e-6926-4170-b130-dfea2801346f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:35.334376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11e17e-6926-4170-b130-dfea2801346f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:35.334735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11e17e-6926-4170-b130-dfea2801346f None admin] PATH_INFO: `/v3/services/network` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:35.335425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d11e17e-6926-4170-b130-dfea2801346f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:35.337707 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d11e17e-6926-4170-b130-dfea2801346f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-6d11e17e-6926-4170-b130-dfea2801346f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:35.354297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:35.355481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:35.355481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:35.355481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:35.355481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:35.355481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:35.355481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:35.355481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:53:35.355481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:53:35.355481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:53:35.355481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:53:35.355481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:35.355481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:35.355481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:35.355481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:35.355481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:35.355481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:35.356344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:35.356344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:35.356344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:35.356344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:35.356344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:35.356344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:35.356344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:35.356344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:35.356344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:35.356344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:35.356344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:53:35.356344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:35.356344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:53:35.356344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:35.356344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 387/774] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:35 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:35.362482 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11e17e-6926-4170-b130-dfea2801346f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:35.403817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:35.417638 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5518278-cfe2-4f35-933b-3d38e2d23bd0 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:35.462480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d612267-35c0-4a0a-ba5d-630f6555f850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:35.463724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d612267-35c0-4a0a-ba5d-630f6555f850 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:35.464220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d612267-35c0-4a0a-ba5d-630f6555f850 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:35.464742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d612267-35c0-4a0a-ba5d-630f6555f850 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:35.465866 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d612267-35c0-4a0a-ba5d-630f6555f850 None admin] RBAC: Adding query filter params (interface=public, service_id=e5d8efc5d8f041579c03bc3f88ce3132, region_id=RegionOne) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:35.466414 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d612267-35c0-4a0a-ba5d-630f6555f850 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:35.468037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d612267-35c0-4a0a-ba5d-630f6555f850 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:35.478114 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 388/775] 10.222.0.83 () {56 vars in 1251 bytes} [Mon Jul 18 14:53:35 2022] GET /identity/v3/endpoints?service_id=e5d8efc5d8f041579c03bc3f88ce3132&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:35.478517 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11e17e-6926-4170-b130-dfea2801346f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:35.485807 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d612267-35c0-4a0a-ba5d-630f6555f850 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:35.518915 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9f033-f200-4065-93fa-f2db40fb2a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:35.519905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9f033-f200-4065-93fa-f2db40fb2a54 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:35.519905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9f033-f200-4065-93fa-f2db40fb2a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:35.520119 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9f033-f200-4065-93fa-f2db40fb2a54 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:35.520860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9f033-f200-4065-93fa-f2db40fb2a54 None admin] RBAC: Adding query filter params (name=network) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:35.521169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9f033-f200-4065-93fa-f2db40fb2a54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:35.522641 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9f033-f200-4065-93fa-f2db40fb2a54 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:35.529622 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 388/776] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Jul 18 14:53:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:35.535429 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9f033-f200-4065-93fa-f2db40fb2a54 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:35.607742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:35.629575 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d612267-35c0-4a0a-ba5d-630f6555f850 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:35.653565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:35.658117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9f033-f200-4065-93fa-f2db40fb2a54 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:35.673368 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50127999-d1b0-48aa-b1da-c42fca538e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:35.674488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50127999-d1b0-48aa-b1da-c42fca538e6e None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:35.675048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50127999-d1b0-48aa-b1da-c42fca538e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:35.675562 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50127999-d1b0-48aa-b1da-c42fca538e6e None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:35.676793 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50127999-d1b0-48aa-b1da-c42fca538e6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:35.678456 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50127999-d1b0-48aa-b1da-c42fca538e6e None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:35.686084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 389/777] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Jul 18 14:53:35 2022] GET /identity/v3/services => generated 772 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:35.701592 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef508af-da09-4c23-9bfa-e379a77515c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:35.702303 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef508af-da09-4c23-9bfa-e379a77515c9 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:35.702627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef508af-da09-4c23-9bfa-e379a77515c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:35.702909 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef508af-da09-4c23-9bfa-e379a77515c9 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:35.703675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef508af-da09-4c23-9bfa-e379a77515c9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:35.703991 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef508af-da09-4c23-9bfa-e379a77515c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:35.705517 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef508af-da09-4c23-9bfa-e379a77515c9 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:35.712621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 389/778] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Jul 18 14:53:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:35.718609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50127999-d1b0-48aa-b1da-c42fca538e6e None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:35.868140 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50127999-d1b0-48aa-b1da-c42fca538e6e None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:35.932626 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a68b8-b52f-4ab7-873c-a5aefbd97e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:35.933425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a68b8-b52f-4ab7-873c-a5aefbd97e7e None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:35.933708 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a68b8-b52f-4ab7-873c-a5aefbd97e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:35.934000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a68b8-b52f-4ab7-873c-a5aefbd97e7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:35.934953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a68b8-b52f-4ab7-873c-a5aefbd97e7e None admin] RBAC: Adding query filter params (interface=public, service_id=cb7fb344e9784f949810ac3bec70f95a, region_id=RegionOne) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:35.935273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a68b8-b52f-4ab7-873c-a5aefbd97e7e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:35.936733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a68b8-b52f-4ab7-873c-a5aefbd97e7e None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:35.948197 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 390/779] 10.222.0.83 () {56 vars in 1251 bytes} [Mon Jul 18 14:53:35 2022] GET /identity/v3/endpoints?service_id=cb7fb344e9784f949810ac3bec70f95a&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:35.954414 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef508af-da09-4c23-9bfa-e379a77515c9 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:36.114431 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef508af-da09-4c23-9bfa-e379a77515c9 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:36.168954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da93ca-f9d5-431e-b959-719cea49e5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:36.169555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da93ca-f9d5-431e-b959-719cea49e5fa None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:36.169839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da93ca-f9d5-431e-b959-719cea49e5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:36.170117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da93ca-f9d5-431e-b959-719cea49e5fa None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:36.170879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da93ca-f9d5-431e-b959-719cea49e5fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:36.172603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da93ca-f9d5-431e-b959-719cea49e5fa None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:36.185522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 390/780] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Jul 18 14:53:35 2022] GET /identity/v3/services => generated 772 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:36.354747 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d34349-6653-4be8-b8e6-3dd58f1cfced None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:36.355204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d34349-6653-4be8-b8e6-3dd58f1cfced None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:36.355324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d34349-6653-4be8-b8e6-3dd58f1cfced None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:36.357227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 391/781] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:36.363433 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf9972-2406-4bd4-90bf-001e101065c6 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:36.363796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf9972-2406-4bd4-90bf-001e101065c6 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:36.363996 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf9972-2406-4bd4-90bf-001e101065c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:36.390470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-62bf9972-2406-4bd4-90bf-001e101065c6 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:36.391710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-62bf9972-2406-4bd4-90bf-001e101065c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:36.427076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:36.497483 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 391/782] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:36.532976 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d34349-6653-4be8-b8e6-3dd58f1cfced None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:36.554347 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:36.666300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d34349-6653-4be8-b8e6-3dd58f1cfced None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:36.722321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ac0e6-940b-4a30-86a1-620a5767b9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:36.723658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ac0e6-940b-4a30-86a1-620a5767b9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:36.724169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ac0e6-940b-4a30-86a1-620a5767b9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:36.724694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ac0e6-940b-4a30-86a1-620a5767b9e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:36.740016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ac0e6-940b-4a30-86a1-620a5767b9e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:36.749229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-2f5ac0e6-940b-4a30-86a1-620a5767b9e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:36.749229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:36.749229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:36.749229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:36.749229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:36.749229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:36.749229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:36.749229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:36.749229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:36.749229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:36.749229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:36.749229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:36.749229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:36.749229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:36.749229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:36.749229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:36.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:36.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:36.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:36.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:36.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:36.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:36.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:36.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:36.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:36.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:36.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:36.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:36.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:36.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:36.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:36.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:36.755425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:36.755425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:36.755425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:36.755425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:36.755425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:36.755425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:36.755425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:36.755425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:36.755425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:36.755425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:36.755425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:36.755425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:36.756496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 392/783] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:36.763003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf9972-2406-4bd4-90bf-001e101065c6 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:36.876963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984d41e-f21a-4e70-acd2-0fc8b24b9824 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:36.877328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984d41e-f21a-4e70-acd2-0fc8b24b9824 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:36.877611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984d41e-f21a-4e70-acd2-0fc8b24b9824 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:36.879275 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 393/784] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:36.892592 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3c921-91f1-48d1-b2f0-71aaf9df77fe None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:36.892960 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3c921-91f1-48d1-b2f0-71aaf9df77fe None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:36.893175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3c921-91f1-48d1-b2f0-71aaf9df77fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:36.910988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf9972-2406-4bd4-90bf-001e101065c6 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:36.934680 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-0cc3c921-91f1-48d1-b2f0-71aaf9df77fe None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:36.935934 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0cc3c921-91f1-48d1-b2f0-71aaf9df77fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:36.961840 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9a76b-41eb-4dac-bc73-8ed5cee66232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:36.963186 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9a76b-41eb-4dac-bc73-8ed5cee66232 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:36.963693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9a76b-41eb-4dac-bc73-8ed5cee66232 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:36.964181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9a76b-41eb-4dac-bc73-8ed5cee66232 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:36.965580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9a76b-41eb-4dac-bc73-8ed5cee66232 None admin] RBAC: Adding query filter params (name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:36.966133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9a76b-41eb-4dac-bc73-8ed5cee66232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:36.968285 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9a76b-41eb-4dac-bc73-8ed5cee66232 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:36.995608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 392/785] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:37.000826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9a76b-41eb-4dac-bc73-8ed5cee66232 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:37.042627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:37.109178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 394/786] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:37.155625 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3c921-91f1-48d1-b2f0-71aaf9df77fe None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:37.159949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:37.177102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9a76b-41eb-4dac-bc73-8ed5cee66232 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:37.245884 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462c058-19a7-4b61-80a1-27709be45082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:37.246707 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462c058-19a7-4b61-80a1-27709be45082 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:37.247395 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462c058-19a7-4b61-80a1-27709be45082 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:37.247395 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462c058-19a7-4b61-80a1-27709be45082 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:37.259917 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462c058-19a7-4b61-80a1-27709be45082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:37.263805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462c058-19a7-4b61-80a1-27709be45082 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:37.280653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-8462c058-19a7-4b61-80a1-27709be45082 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:37.280653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:37.280653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:37.280653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:37.280653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:37.280653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:37.280653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:37.280653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:37.280653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:37.280653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:37.280653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:37.280653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:37.280653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:37.280653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:37.280653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:37.281703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:37.282714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:37.282714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:37.282714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:37.282714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:37.282714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 393/787] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:37.291699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462c058-19a7-4b61-80a1-27709be45082 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:37.361748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3c921-91f1-48d1-b2f0-71aaf9df77fe None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:37.462974 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:37.516959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462c058-19a7-4b61-80a1-27709be45082 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:37.577130 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8926ff8-f21b-4fda-b33d-ad0bb0ed57d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:37.577811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8926ff8-f21b-4fda-b33d-ad0bb0ed57d7 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:37.578101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8926ff8-f21b-4fda-b33d-ad0bb0ed57d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:37.578386 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8926ff8-f21b-4fda-b33d-ad0bb0ed57d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:37.579235 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8926ff8-f21b-4fda-b33d-ad0bb0ed57d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:37.579557 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8926ff8-f21b-4fda-b33d-ad0bb0ed57d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:37.581056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8926ff8-f21b-4fda-b33d-ad0bb0ed57d7 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:37.594671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 394/788] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:37.599886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8926ff8-f21b-4fda-b33d-ad0bb0ed57d7 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:37.614038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59485086-4d04-4aa7-babf-eb3a28e0837c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:37.615207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59485086-4d04-4aa7-babf-eb3a28e0837c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:37.615672 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59485086-4d04-4aa7-babf-eb3a28e0837c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:37.616121 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59485086-4d04-4aa7-babf-eb3a28e0837c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:37.642476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59485086-4d04-4aa7-babf-eb3a28e0837c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:37.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-59485086-4d04-4aa7-babf-eb3a28e0837c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:53:37.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:37.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:37.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:37.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:37.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:37.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:37.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:37.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:37.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:37.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:37.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:37.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:37.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:37.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:37.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:37.652564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:37.652564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:37.652564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:37.652564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:37.652564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:37.652564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:37.652564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:37.652564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:37.652564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:37.652564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:37.652564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:37.652564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:37.652564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:37.652564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:37.652564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:37.652564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:37.653633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:37.653633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:37.653633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:37.653633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:37.653633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:37.653633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:37.653633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:37.653633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:37.653633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:37.653633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:37.653633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:53:37.653633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:37.656321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 395/789] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:53:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 501 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:37.662255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59485086-4d04-4aa7-babf-eb3a28e0837c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:37.729035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8926ff8-f21b-4fda-b33d-ad0bb0ed57d7 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:37.765655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:37.771919 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b705c-d3d2-47f3-a756-d36578ea31f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:37.772683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b705c-d3d2-47f3-a756-d36578ea31f2 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:37.773001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b705c-d3d2-47f3-a756-d36578ea31f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:37.773316 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b705c-d3d2-47f3-a756-d36578ea31f2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:37.782859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b705c-d3d2-47f3-a756-d36578ea31f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:37.784914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b705c-d3d2-47f3-a756-d36578ea31f2 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:37.803931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-2a6b705c-d3d2-47f3-a756-d36578ea31f2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:53:37.803931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:37.803931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:37.803931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:37.803931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:37.803931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:37.803931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:37.803931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:37.803931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:37.803931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:37.803931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:37.803931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:37.803931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:37.803931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:37.803931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:37.803931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:37.806044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:37.807078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:37.807078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:37.807078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:37.807078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:37.807078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:37.807078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:37.807078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:37.807078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:37.807078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:53:37.807078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:37.807078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 395/790] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Jul 18 14:53:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:37.811991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:37.815600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b705c-d3d2-47f3-a756-d36578ea31f2 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:37.879724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59485086-4d04-4aa7-babf-eb3a28e0837c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:37.941491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1c47e-cffb-4b92-97dd-5dde5045ee5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:37.944209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1c47e-cffb-4b92-97dd-5dde5045ee5d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:37.947101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b705c-d3d2-47f3-a756-d36578ea31f2 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:37.947705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1c47e-cffb-4b92-97dd-5dde5045ee5d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:37.948227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1c47e-cffb-4b92-97dd-5dde5045ee5d None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:37.949222 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1c47e-cffb-4b92-97dd-5dde5045ee5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:37.949750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1c47e-cffb-4b92-97dd-5dde5045ee5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:37.951399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1c47e-cffb-4b92-97dd-5dde5045ee5d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:37.973484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 396/791] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:53:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:37.980962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1c47e-cffb-4b92-97dd-5dde5045ee5d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:38.039856 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d053a13-dee2-4fb8-8a0d-836b1cfda145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:38.041518 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d053a13-dee2-4fb8-8a0d-836b1cfda145 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:38.042031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d053a13-dee2-4fb8-8a0d-836b1cfda145 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:38.042524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d053a13-dee2-4fb8-8a0d-836b1cfda145 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:38.043645 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d053a13-dee2-4fb8-8a0d-836b1cfda145 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:38.044281 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d053a13-dee2-4fb8-8a0d-836b1cfda145 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:38.046100 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d053a13-dee2-4fb8-8a0d-836b1cfda145 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:38.074482 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 396/792] 10.222.0.83 () {56 vars in 1149 bytes} [Mon Jul 18 14:53:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:38.095637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d053a13-dee2-4fb8-8a0d-836b1cfda145 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:38.126673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1c47e-cffb-4b92-97dd-5dde5045ee5d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:38.159631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:38.164779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd573bbf-d1b1-4ff8-a7a0-3bc1dddee1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:38.165611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd573bbf-d1b1-4ff8-a7a0-3bc1dddee1af None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:38.165895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd573bbf-d1b1-4ff8-a7a0-3bc1dddee1af None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:38.166167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd573bbf-d1b1-4ff8-a7a0-3bc1dddee1af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:38.181569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd573bbf-d1b1-4ff8-a7a0-3bc1dddee1af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:38.184266 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd573bbf-d1b1-4ff8-a7a0-3bc1dddee1af None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:38.189362 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:38.214694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-bd573bbf-d1b1-4ff8-a7a0-3bc1dddee1af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:38.214694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:38.214694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:38.214694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:38.214694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:38.214694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:38.214694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:38.214694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:38.214694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:38.214694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:38.214694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:38.214694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:38.214694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:38.214694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:38.214694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:38.215739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:38.216749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:38.216749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:38.216749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:38.216749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:38.224423 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 397/793] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:38.225946 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94faa8f2-05b9-4d65-8e61-8b581960ec31 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:38.226318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94faa8f2-05b9-4d65-8e61-8b581960ec31 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:38.227024 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94faa8f2-05b9-4d65-8e61-8b581960ec31 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:38.232792 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 398/794] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:38.232792 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94faa8f2-05b9-4d65-8e61-8b581960ec31 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:38.267304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d053a13-dee2-4fb8-8a0d-836b1cfda145 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:38.314196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f36627-79e5-4a87-bcc1-af477237ecff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:38.315107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f36627-79e5-4a87-bcc1-af477237ecff None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:38.315554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f36627-79e5-4a87-bcc1-af477237ecff None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:38.315920 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f36627-79e5-4a87-bcc1-af477237ecff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:38.328826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f36627-79e5-4a87-bcc1-af477237ecff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:38.330709 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f36627-79e5-4a87-bcc1-af477237ecff None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:38.351050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-18f36627-79e5-4a87-bcc1-af477237ecff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:38.351050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:38.351050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:38.351050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:38.351050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:38.351050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:38.351050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:38.351050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:38.351050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:38.351050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:38.351050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:38.351050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:38.351050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:38.351050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:38.351050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:38.353499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:38.355082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:38.355082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:38.355082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:38.355082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:38.361391 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 397/795] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:38.361391 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ceb68-913f-4c50-9677-0877fdd21cfe None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:38.361391 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ceb68-913f-4c50-9677-0877fdd21cfe None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:38.361391 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ceb68-913f-4c50-9677-0877fdd21cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:38.369211 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94faa8f2-05b9-4d65-8e61-8b581960ec31 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:38.384738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-d30ceb68-913f-4c50-9677-0877fdd21cfe None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:38.387087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d30ceb68-913f-4c50-9677-0877fdd21cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:38.418239 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d59cd-8677-46bb-a9c3-844b85b51442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:38.420276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d59cd-8677-46bb-a9c3-844b85b51442 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:38.420742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d59cd-8677-46bb-a9c3-844b85b51442 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:38.421187 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d59cd-8677-46bb-a9c3-844b85b51442 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:38.422081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d59cd-8677-46bb-a9c3-844b85b51442 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:38.422614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d59cd-8677-46bb-a9c3-844b85b51442 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:38.424189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d59cd-8677-46bb-a9c3-844b85b51442 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:38.437574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 399/796] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:38.439029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d59cd-8677-46bb-a9c3-844b85b51442 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:38.483421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:38.505981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 398/797] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:38.509376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548cf44-ff59-4070-874f-8804f30673c9 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:38.509897 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548cf44-ff59-4070-874f-8804f30673c9 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:38.510413 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548cf44-ff59-4070-874f-8804f30673c9 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:38.520147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 399/798] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:38 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:38.521692 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4548cf44-ff59-4070-874f-8804f30673c9 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:38.568357 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:38.570728 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d59cd-8677-46bb-a9c3-844b85b51442 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:38.609361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d88fb-bfe2-4424-8b8a-297b1aeea52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:38.610059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d88fb-bfe2-4424-8b8a-297b1aeea52c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:38.610323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d88fb-bfe2-4424-8b8a-297b1aeea52c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:38.610636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d88fb-bfe2-4424-8b8a-297b1aeea52c None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:38.611375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d88fb-bfe2-4424-8b8a-297b1aeea52c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:38.611685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d88fb-bfe2-4424-8b8a-297b1aeea52c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:38.613138 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d88fb-bfe2-4424-8b8a-297b1aeea52c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:38.632374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 400/799] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:38.632374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682cc5c-40b0-4b8d-823e-00c809c70218 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:38.632374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682cc5c-40b0-4b8d-823e-00c809c70218 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:38.632374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682cc5c-40b0-4b8d-823e-00c809c70218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:38.650109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b682cc5c-40b0-4b8d-823e-00c809c70218 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:38.651345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b682cc5c-40b0-4b8d-823e-00c809c70218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:38.664703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4548cf44-ff59-4070-874f-8804f30673c9 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:38.705260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcef72-0946-47d3-a984-b4bd86744266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:38.706065 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcef72-0946-47d3-a984-b4bd86744266 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:38.706352 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcef72-0946-47d3-a984-b4bd86744266 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:38.706654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcef72-0946-47d3-a984-b4bd86744266 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:38.718949 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bcef72-0946-47d3-a984-b4bd86744266 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:38.720431 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bcef72-0946-47d3-a984-b4bd86744266 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:38.730239 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 401/800] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:38.731143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682cc5c-40b0-4b8d-823e-00c809c70218 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:38.736498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-85bcef72-0946-47d3-a984-b4bd86744266 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:53:38.736498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:38.736498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:38.736498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:38.736498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:38.736498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:38.736498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:38.736498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:38.736498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:38.736498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:38.736498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:38.736498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:38.736498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:38.736498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:38.736498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:38.736498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:38.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:38.739113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:38.739113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:38.739113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:38.739113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:38.739113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:38.739113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:38.739113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:38.739113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:38.739113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:53:38.739113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:38.739113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 400/801] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Jul 18 14:53:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:38.739113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcef72-0946-47d3-a984-b4bd86744266 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:38.768879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:38.844400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:38.854039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcef72-0946-47d3-a984-b4bd86744266 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:38.855428 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682cc5c-40b0-4b8d-823e-00c809c70218 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:38.893360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0aec1-4f26-4b90-9d12-d9e6d07a9f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:38.894281 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0aec1-4f26-4b90-9d12-d9e6d07a9f41 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:38.894668 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0aec1-4f26-4b90-9d12-d9e6d07a9f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:38.895019 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0aec1-4f26-4b90-9d12-d9e6d07a9f41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:38.898650 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab644d61-2feb-4408-8eef-4f836b449939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:38.899540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab644d61-2feb-4408-8eef-4f836b449939 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:38.899839 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab644d61-2feb-4408-8eef-4f836b449939 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:38.900349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab644d61-2feb-4408-8eef-4f836b449939 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:38.902916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0aec1-4f26-4b90-9d12-d9e6d07a9f41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:38.904478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0aec1-4f26-4b90-9d12-d9e6d07a9f41 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:38.915901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab644d61-2feb-4408-8eef-4f836b449939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:38.919586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-fda0aec1-4f26-4b90-9d12-d9e6d07a9f41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:38.919586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:38.919586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:38.919586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:38.919586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:38.919586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:38.919586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:38.919586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:38.919586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:38.919586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:38.919586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:38.919586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:38.919586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:38.919586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:38.919586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:38.920812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:38.922295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:38.922295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:38.922295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:38.922295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:38.922295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:38.922295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:38.922295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:38.922295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:38.923257 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 401/802] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:38.924337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b59c92-58ab-4cdf-8cf1-ad7c67f27606 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:38.924686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b59c92-58ab-4cdf-8cf1-ad7c67f27606 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:38.925037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b59c92-58ab-4cdf-8cf1-ad7c67f27606 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:38.925255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-ab644d61-2feb-4408-8eef-4f836b449939 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:53:38.925255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:38.925255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:38.925255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:38.925255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:38.925255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:38.925255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:38.925255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:38.925255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:38.925255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:38.925255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:38.925255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:38.925255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:38.925255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:38.925255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:38.925255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:38.933643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:38.933643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:38.933643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:38.933643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:38.933643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:38.933643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:38.933643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:38.933643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:38.933643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:38.933643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:38.933643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:38.933643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:38.933643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:38.933643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:38.933643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:38.933643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:38.935497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:38.935497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:38.935497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:38.935497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:38.935497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:38.935497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:38.935497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:38.935497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:38.935497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:38.935497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:38.935497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:53:38.935497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:38.935497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 402/803] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:53:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:38.935497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab644d61-2feb-4408-8eef-4f836b449939 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:38.936567 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 402/804] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:38.936567 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b59c92-58ab-4cdf-8cf1-ad7c67f27606 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:39.028046 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:39.045134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab644d61-2feb-4408-8eef-4f836b449939 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:39.046065 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b59c92-58ab-4cdf-8cf1-ad7c67f27606 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:39.082892 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc573380-e1c9-49de-96d1-4c731229d6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:39.083581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc573380-e1c9-49de-96d1-4c731229d6bc None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:39.083870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc573380-e1c9-49de-96d1-4c731229d6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:39.083974 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b96ebc-ddf9-4d75-8ad0-f352c9dcf84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:39.084151 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc573380-e1c9-49de-96d1-4c731229d6bc None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:39.084911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b96ebc-ddf9-4d75-8ad0-f352c9dcf84d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:39.085033 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc573380-e1c9-49de-96d1-4c731229d6bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:39.085297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc573380-e1c9-49de-96d1-4c731229d6bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:39.085397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b96ebc-ddf9-4d75-8ad0-f352c9dcf84d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:39.085744 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b96ebc-ddf9-4d75-8ad0-f352c9dcf84d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:39.086400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b96ebc-ddf9-4d75-8ad0-f352c9dcf84d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:39.086833 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc573380-e1c9-49de-96d1-4c731229d6bc None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:39.087893 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b96ebc-ddf9-4d75-8ad0-f352c9dcf84d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:39.104600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-54b96ebc-ddf9-4d75-8ad0-f352c9dcf84d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:53:39.104600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:39.104600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:53:39.104600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:53:39.104600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:53:39.104600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:53:39.104600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:53:39.104600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:39.104600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:53:39.104600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:39.104600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:39.104600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:39.104600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:39.104600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:39.104600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:39.104600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:39.105882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:39.105882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:39.105882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:39.105882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:39.105882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:39.105882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:39.105882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:39.105882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:53:39.105882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:53:39.105882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:53:39.105882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:53:39.105882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:39.105882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:39.105882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:39.105882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:39.105882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:39.107087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:39.107087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:39.107087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:39.107087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:39.107087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:39.107087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:39.107087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:39.107087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:39.107087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:39.107087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:39.107087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:39.107087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:53:39.107087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:39.107087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:53:39.107087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:39.107087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 403/805] 10.222.0.83 () {56 vars in 1129 bytes} [Mon Jul 18 14:53:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:39.108336 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b96ebc-ddf9-4d75-8ad0-f352c9dcf84d None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:39.108409 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 403/806] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Jul 18 14:53:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:39.108945 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4469ab-bdd4-40f9-828f-57c76ffae65e None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:39.109286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4469ab-bdd4-40f9-828f-57c76ffae65e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:39.109609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4469ab-bdd4-40f9-828f-57c76ffae65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:39.127086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-8e4469ab-bdd4-40f9-828f-57c76ffae65e None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:39.127634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:39.128335 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-8e4469ab-bdd4-40f9-828f-57c76ffae65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:39.198442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 404/807] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:39.199499 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4469ab-bdd4-40f9-828f-57c76ffae65e None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:39.217984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b96ebc-ddf9-4d75-8ad0-f352c9dcf84d None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:39.260565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b391212-66c0-4980-b4e6-992e4b7e69fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:39.261235 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b391212-66c0-4980-b4e6-992e4b7e69fe None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:39.261522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b391212-66c0-4980-b4e6-992e4b7e69fe None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:39.261802 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b391212-66c0-4980-b4e6-992e4b7e69fe None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:39.262639 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b391212-66c0-4980-b4e6-992e4b7e69fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:39.262940 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b391212-66c0-4980-b4e6-992e4b7e69fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:39.264417 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b391212-66c0-4980-b4e6-992e4b7e69fe None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:39.277378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 404/808] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:39.278230 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b391212-66c0-4980-b4e6-992e4b7e69fe None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:39.287471 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:39.315705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4469ab-bdd4-40f9-828f-57c76ffae65e None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:39.353977 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c46970-79ba-42ae-9228-3af7c34f343e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:39.354740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c46970-79ba-42ae-9228-3af7c34f343e None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:39.354958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c46970-79ba-42ae-9228-3af7c34f343e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:39.355231 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c46970-79ba-42ae-9228-3af7c34f343e None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:39.355975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c46970-79ba-42ae-9228-3af7c34f343e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:39.356289 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c46970-79ba-42ae-9228-3af7c34f343e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:39.357923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c46970-79ba-42ae-9228-3af7c34f343e None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:39.362027 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:39.367594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 405/809] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:53:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:39.368797 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c46970-79ba-42ae-9228-3af7c34f343e None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:39.389676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b391212-66c0-4980-b4e6-992e4b7e69fe None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:39.439062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e42536-2f5a-4fdd-8c6c-914c77c41187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:39.440204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e42536-2f5a-4fdd-8c6c-914c77c41187 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:39.440671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e42536-2f5a-4fdd-8c6c-914c77c41187 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:39.441125 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e42536-2f5a-4fdd-8c6c-914c77c41187 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:39.442169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e42536-2f5a-4fdd-8c6c-914c77c41187 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:39.442486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e42536-2f5a-4fdd-8c6c-914c77c41187 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:39.443948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e42536-2f5a-4fdd-8c6c-914c77c41187 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:39.449267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 405/810] 10.222.0.83 () {56 vars in 1124 bytes} [Mon Jul 18 14:53:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:39.450117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e42536-2f5a-4fdd-8c6c-914c77c41187 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:39.474493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c46970-79ba-42ae-9228-3af7c34f343e None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:39.511106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65fc05-3824-4d7f-8eb3-f94f13570947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:39.511762 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65fc05-3824-4d7f-8eb3-f94f13570947 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:39.512131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65fc05-3824-4d7f-8eb3-f94f13570947 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:39.512309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65fc05-3824-4d7f-8eb3-f94f13570947 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:39.515948 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:39.561008 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e42536-2f5a-4fdd-8c6c-914c77c41187 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:39.588054 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65fc05-3824-4d7f-8eb3-f94f13570947 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:39.589528 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65fc05-3824-4d7f-8eb3-f94f13570947 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:39.603067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566d007-fb9e-4a4e-9153-674b605200af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:39.604135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566d007-fb9e-4a4e-9153-674b605200af None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:39.604495 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566d007-fb9e-4a4e-9153-674b605200af None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:39.604900 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566d007-fb9e-4a4e-9153-674b605200af None admin] PATH_INFO: `/v3/services/network` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:39.605589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566d007-fb9e-4a4e-9153-674b605200af None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:39.607365 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566d007-fb9e-4a4e-9153-674b605200af None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-e566d007-fb9e-4a4e-9153-674b605200af None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:39.619791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:39.620879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:39.620879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:39.620879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:39.620879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:39.620879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:39.620879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:39.620879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:53:39.620879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:53:39.620879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:53:39.620879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:53:39.620879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:39.620879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:39.620879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:39.620879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:39.620879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:39.620879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:39.621731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:39.621731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:39.621731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:39.621731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:39.621731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:39.621731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:39.621731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:39.621731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:39.621731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:39.621731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:39.621731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:53:39.621731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:39.621731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:53:39.621731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:39.621731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 406/811] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:39 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:39.622953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566d007-fb9e-4a4e-9153-674b605200af None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:39.628767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:39.700325 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 406/812] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:53:39.701234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65fc05-3824-4d7f-8eb3-f94f13570947 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:39.738432 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566d007-fb9e-4a4e-9153-674b605200af None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:39.742992 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:39.776677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bea3e-ba05-4ef2-a1e2-65e37efdac67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:39.777582 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bea3e-ba05-4ef2-a1e2-65e37efdac67 None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:39.777867 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bea3e-ba05-4ef2-a1e2-65e37efdac67 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:39.778148 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bea3e-ba05-4ef2-a1e2-65e37efdac67 None admin] PATH_INFO: `/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/692fef632836474da09f0e8a009ffa4e/roles/29b8b950372848ffa9a75e778ee66b4c` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:39.793766 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2bea3e-ba05-4ef2-a1e2-65e37efdac67 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f777cd6a0c744e594f8e29ab58cf8bd, user_id=692fef632836474da09f0e8a009ffa4e, role_id=29b8b950372848ffa9a75e778ee66b4c)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:39.795288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2bea3e-ba05-4ef2-a1e2-65e37efdac67 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:39.816025 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65fc05-3824-4d7f-8eb3-f94f13570947 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:39.852423 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc8de0-45ca-498c-a75f-19627b499677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:39.853277 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc8de0-45ca-498c-a75f-19627b499677 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:39.853566 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc8de0-45ca-498c-a75f-19627b499677 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:39.853846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc8de0-45ca-498c-a75f-19627b499677 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:39.864664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc8de0-45ca-498c-a75f-19627b499677 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:39.866053 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc8de0-45ca-498c-a75f-19627b499677 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:39.881021 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 407/813] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:53:39 2022] PUT /identity/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/692fef632836474da09f0e8a009ffa4e/roles/29b8b950372848ffa9a75e778ee66b4c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:53:39.882125 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bea3e-ba05-4ef2-a1e2-65e37efdac67 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:39.889088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-07fc8de0-45ca-498c-a75f-19627b499677 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:39.889088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:39.889088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:39.889088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:39.889088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:39.889088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:39.889088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:39.889088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:39.889088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:39.889088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:39.889088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:39.889088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:39.889088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:39.889088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:39.889088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:39.894359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:39.895561 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:39.895561 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:39.895561 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:39.895561 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:39.895561 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 407/814] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:39.895561 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc8de0-45ca-498c-a75f-19627b499677 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:39.964720 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:40.024393 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:40.040453 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bea3e-ba05-4ef2-a1e2-65e37efdac67 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:40.042107 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc8de0-45ca-498c-a75f-19627b499677 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:40.086762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659c17b-3502-407c-b087-026217c7d61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:40.087542 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c17b-3502-407c-b087-026217c7d61b None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:40.087788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c17b-3502-407c-b087-026217c7d61b None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:40.088156 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c17b-3502-407c-b087-026217c7d61b None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:40.088794 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659c17b-3502-407c-b087-026217c7d61b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:40.089114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659c17b-3502-407c-b087-026217c7d61b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:40.090274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c779a3-b341-4a94-a111-ca9c7e3513db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:40.090632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659c17b-3502-407c-b087-026217c7d61b None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:40.091057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c779a3-b341-4a94-a111-ca9c7e3513db None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:40.091370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c779a3-b341-4a94-a111-ca9c7e3513db None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:40.091652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c779a3-b341-4a94-a111-ca9c7e3513db None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:40.092385 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c779a3-b341-4a94-a111-ca9c7e3513db None admin] RBAC: Adding query filter params (name=network) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:40.092695 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c779a3-b341-4a94-a111-ca9c7e3513db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:40.094087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c779a3-b341-4a94-a111-ca9c7e3513db None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:40.096763 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 408/815] 10.222.0.83 () {56 vars in 1124 bytes} [Mon Jul 18 14:53:39 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:40.099693 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 408/816] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Jul 18 14:53:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:40.099983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659c17b-3502-407c-b087-026217c7d61b None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:40.100599 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c779a3-b341-4a94-a111-ca9c7e3513db None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:40.223563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659c17b-3502-407c-b087-026217c7d61b None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:40.228402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c779a3-b341-4a94-a111-ca9c7e3513db None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:40.244213 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:40.267956 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7536b4-31cd-496a-a5c3-0e74d63738b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:40.268764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7536b4-31cd-496a-a5c3-0e74d63738b6 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:40.269047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7536b4-31cd-496a-a5c3-0e74d63738b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:40.269324 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7536b4-31cd-496a-a5c3-0e74d63738b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:40.271106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573634bb-6280-47b8-8c14-076f65e8e5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:40.272010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573634bb-6280-47b8-8c14-076f65e8e5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:40.272695 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573634bb-6280-47b8-8c14-076f65e8e5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:40.273506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573634bb-6280-47b8-8c14-076f65e8e5d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:40.274320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573634bb-6280-47b8-8c14-076f65e8e5d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:40.274630 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573634bb-6280-47b8-8c14-076f65e8e5d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:40.276091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573634bb-6280-47b8-8c14-076f65e8e5d5 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:40.281912 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7536b4-31cd-496a-a5c3-0e74d63738b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:40.284823 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7536b4-31cd-496a-a5c3-0e74d63738b6 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:40.294752 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 409/817] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:40.295600 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573634bb-6280-47b8-8c14-076f65e8e5d5 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:40.309476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-3f7536b4-31cd-496a-a5c3-0e74d63738b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:40.309476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:40.309476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:40.309476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:40.309476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:40.309476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:40.309476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:40.309476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:40.309476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:40.309476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:40.309476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:40.309476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:40.309476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:40.309476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:40.309476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:40.310679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:40.311795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:40.311795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:40.311795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:40.311795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:40.315270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:40.316332 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 409/818] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:40.317595 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7536b4-31cd-496a-a5c3-0e74d63738b6 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:40.416779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573634bb-6280-47b8-8c14-076f65e8e5d5 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:40.437042 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7536b4-31cd-496a-a5c3-0e74d63738b6 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:40.456544 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83038c0e-98f5-46b9-a03d-59b30d83cd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:40.457327 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83038c0e-98f5-46b9-a03d-59b30d83cd8b None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:40.457617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83038c0e-98f5-46b9-a03d-59b30d83cd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:40.457896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83038c0e-98f5-46b9-a03d-59b30d83cd8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:40.458695 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83038c0e-98f5-46b9-a03d-59b30d83cd8b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:40.460222 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83038c0e-98f5-46b9-a03d-59b30d83cd8b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:40.475768 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b7ef4-584e-406e-aa26-e7e04207cf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:40.476580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b7ef4-584e-406e-aa26-e7e04207cf58 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:40.477150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b7ef4-584e-406e-aa26-e7e04207cf58 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:40.477567 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b7ef4-584e-406e-aa26-e7e04207cf58 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:40.478365 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b7ef4-584e-406e-aa26-e7e04207cf58 None admin] RBAC: Adding query filter params (type=network) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:40.478707 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b7ef4-584e-406e-aa26-e7e04207cf58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:40.480157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b7ef4-584e-406e-aa26-e7e04207cf58 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:40.486124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 410/819] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Jul 18 14:53:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:40.487036 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b7ef4-584e-406e-aa26-e7e04207cf58 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:40.520073 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:40.575600 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 410/820] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Jul 18 14:53:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:53:40.576004 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83038c0e-98f5-46b9-a03d-59b30d83cd8b None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:40.607084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b7ef4-584e-406e-aa26-e7e04207cf58 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:40.658100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:40.665776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed25c8-7e6d-4d6b-8651-e4af6c871e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:40.667798 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed25c8-7e6d-4d6b-8651-e4af6c871e79 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:40.667798 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed25c8-7e6d-4d6b-8651-e4af6c871e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:40.668217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed25c8-7e6d-4d6b-8651-e4af6c871e79 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:40.678499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed25c8-7e6d-4d6b-8651-e4af6c871e79 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:40.680953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed25c8-7e6d-4d6b-8651-e4af6c871e79 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:40.703767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-c0ed25c8-7e6d-4d6b-8651-e4af6c871e79 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:53:40.703767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:40.703767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:40.703767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:40.703767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:40.703767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:40.703767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:40.703767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:40.703767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:40.703767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:40.703767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:40.703767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:40.703767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:40.703767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:40.703767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:40.703767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:40.704817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:40.705808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:40.705808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:40.705808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:40.705808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:40.705808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:40.705808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:40.705808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:40.705808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:40.705808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:53:40.705808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:40.706375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83038c0e-98f5-46b9-a03d-59b30d83cd8b None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:40.710218 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 411/821] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:53:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:40.712331 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed25c8-7e6d-4d6b-8651-e4af6c871e79 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:40.756566 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460dccea-0f36-4d16-8c79-158fe246c569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:40.757485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460dccea-0f36-4d16-8c79-158fe246c569 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:40.757889 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460dccea-0f36-4d16-8c79-158fe246c569 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:40.758289 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460dccea-0f36-4d16-8c79-158fe246c569 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:40.759209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460dccea-0f36-4d16-8c79-158fe246c569 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:40.759634 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460dccea-0f36-4d16-8c79-158fe246c569 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:40.761269 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460dccea-0f36-4d16-8c79-158fe246c569 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:40.778744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 411/822] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:40.779778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460dccea-0f36-4d16-8c79-158fe246c569 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:40.854561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:40.878155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed25c8-7e6d-4d6b-8651-e4af6c871e79 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:40.917737 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460dccea-0f36-4d16-8c79-158fe246c569 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:40.927557 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34dae90-c7a7-4c16-8042-e3a43742afc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:40.928424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dae90-c7a7-4c16-8042-e3a43742afc9 None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:40.928886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dae90-c7a7-4c16-8042-e3a43742afc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:40.929869 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dae90-c7a7-4c16-8042-e3a43742afc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:40.930807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34dae90-c7a7-4c16-8042-e3a43742afc9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:40.933189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34dae90-c7a7-4c16-8042-e3a43742afc9 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:40.955148 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:40.963760 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffe591-e98e-4ffa-9675-c3151c5e0106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:40.964576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffe591-e98e-4ffa-9675-c3151c5e0106 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:40.964977 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffe591-e98e-4ffa-9675-c3151c5e0106 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:40.965330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffe591-e98e-4ffa-9675-c3151c5e0106 None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:40.966337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffe591-e98e-4ffa-9675-c3151c5e0106 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:40.966739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffe591-e98e-4ffa-9675-c3151c5e0106 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:40.968453 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffe591-e98e-4ffa-9675-c3151c5e0106 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:40.989544 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 412/823] 10.222.0.83 () {56 vars in 1135 bytes} [Mon Jul 18 14:53:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:40.994579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffe591-e98e-4ffa-9675-c3151c5e0106 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:41.031809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 412/824] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Jul 18 14:53:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:53:41.032876 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34dae90-c7a7-4c16-8042-e3a43742afc9 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:41.160013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffe591-e98e-4ffa-9675-c3151c5e0106 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:41.224778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34dae90-c7a7-4c16-8042-e3a43742afc9 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:41.247660 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be57736-41be-45cd-8a0d-e1c120114478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:41.248511 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be57736-41be-45cd-8a0d-e1c120114478 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:41.248801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be57736-41be-45cd-8a0d-e1c120114478 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:41.249079 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be57736-41be-45cd-8a0d-e1c120114478 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:41.262653 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be57736-41be-45cd-8a0d-e1c120114478 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:41.266061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be57736-41be-45cd-8a0d-e1c120114478 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:41.275940 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13317a6c-527f-49a5-bd9b-14df99251356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:41.277167 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13317a6c-527f-49a5-bd9b-14df99251356 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:41.277719 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13317a6c-527f-49a5-bd9b-14df99251356 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:41.278211 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13317a6c-527f-49a5-bd9b-14df99251356 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:41.285901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-2be57736-41be-45cd-8a0d-e1c120114478 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:41.285901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:41.285901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:41.285901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:41.285901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:41.285901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:41.285901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:41.285901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:41.285901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:41.285901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:41.285901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:41.285901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:41.285901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:41.285901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:41.285901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:41.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:41.288099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:41.288099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:41.288099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:41.288099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:41.288099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:41.288099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:41.288099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:41.288099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:41.288099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 413/825] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:41.292757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:41.296326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be57736-41be-45cd-8a0d-e1c120114478 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:41.297920 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13317a6c-527f-49a5-bd9b-14df99251356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:41.298979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13317a6c-527f-49a5-bd9b-14df99251356 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:41.321259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-13317a6c-527f-49a5-bd9b-14df99251356 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:41.321259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:41.321259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:41.321259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:41.321259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:41.321259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:41.321259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:41.321259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:41.321259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:41.321259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:41.321259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:41.321259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:41.321259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:41.321259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:41.321259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:41.322369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:41.323335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:41.323335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:41.323335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:41.323335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:41.328134 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 413/826] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:41.329451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13317a6c-527f-49a5-bd9b-14df99251356 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:41.374993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:41.440614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be57736-41be-45cd-8a0d-e1c120114478 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:41.492940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c3861-976b-4e97-bcf3-569a626d3450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:41.494121 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c3861-976b-4e97-bcf3-569a626d3450 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:41.497271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c3861-976b-4e97-bcf3-569a626d3450 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:41.497913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c3861-976b-4e97-bcf3-569a626d3450 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:41.499390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c3861-976b-4e97-bcf3-569a626d3450 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:41.499934 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c3861-976b-4e97-bcf3-569a626d3450 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:41.501803 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c3861-976b-4e97-bcf3-569a626d3450 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:41.518565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 414/827] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:41.525010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c3861-976b-4e97-bcf3-569a626d3450 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:41.595175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13317a6c-527f-49a5-bd9b-14df99251356 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:41.634787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17d886-d637-49c3-ac38-ddbab7c565cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:41.635575 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17d886-d637-49c3-ac38-ddbab7c565cd None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:41.635938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17d886-d637-49c3-ac38-ddbab7c565cd None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:41.636532 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17d886-d637-49c3-ac38-ddbab7c565cd None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:41.637090 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17d886-d637-49c3-ac38-ddbab7c565cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:41.637498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17d886-d637-49c3-ac38-ddbab7c565cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:41.639020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17d886-d637-49c3-ac38-ddbab7c565cd None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:41.656015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 414/828] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:41.657503 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17d886-d637-49c3-ac38-ddbab7c565cd None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:41.665685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c3861-976b-4e97-bcf3-569a626d3450 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:41.677063 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:41.716329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:41.716960 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9efed0-5207-4604-996a-9a16405f48e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:41.718874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9efed0-5207-4604-996a-9a16405f48e4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:41.719358 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9efed0-5207-4604-996a-9a16405f48e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:41.720014 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9efed0-5207-4604-996a-9a16405f48e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:41.721751 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9efed0-5207-4604-996a-9a16405f48e4 None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=7e1355d18fb049468651b3675c217d60) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:41.722284 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9efed0-5207-4604-996a-9a16405f48e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:41.724603 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9efed0-5207-4604-996a-9a16405f48e4 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:41.734816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 415/829] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:41 2022] GET /identity/v3/role_assignments?user.id=7e1355d18fb049468651b3675c217d60&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=5891920310b54787bd1152d736d2a3f1 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:41.781952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17d886-d637-49c3-ac38-ddbab7c565cd None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:41.828151 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ee3a7-d5b3-4388-800f-5b99bc8e7fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:41.829411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ee3a7-d5b3-4388-800f-5b99bc8e7fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:41.829885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ee3a7-d5b3-4388-800f-5b99bc8e7fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:41.830376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ee3a7-d5b3-4388-800f-5b99bc8e7fd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:41.838957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ee3a7-d5b3-4388-800f-5b99bc8e7fd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:41.840819 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ee3a7-d5b3-4388-800f-5b99bc8e7fd7 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:41.857979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-f15ee3a7-d5b3-4388-800f-5b99bc8e7fd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:41.857979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:41.857979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:41.857979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:41.857979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:41.857979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:41.857979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:41.857979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:41.857979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:41.857979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:41.857979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:41.857979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:41.857979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:41.857979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:41.857979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:41.859145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:41.860059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:41.860059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:41.860059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:41.860059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:41.860059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:41.860059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:41.860059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:41.860059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:41.862623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 415/830] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:41.871839 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9efed0-5207-4604-996a-9a16405f48e4 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:42.013563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9efed0-5207-4604-996a-9a16405f48e4 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:42.050393 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a6a73-b6e8-49c9-be74-62798a144e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:42.051416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a6a73-b6e8-49c9-be74-62798a144e18 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:42.051854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a6a73-b6e8-49c9-be74-62798a144e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:42.052286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a6a73-b6e8-49c9-be74-62798a144e18 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:42.053309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a6a73-b6e8-49c9-be74-62798a144e18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:42.053769 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a6a73-b6e8-49c9-be74-62798a144e18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:42.056098 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a6a73-b6e8-49c9-be74-62798a144e18 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:42.068924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 416/831] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:42.075504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ee3a7-d5b3-4388-800f-5b99bc8e7fd7 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:42.144489 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82f6c1-98c3-4be1-93e0-eaaf4960ed20 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:42.144685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82f6c1-98c3-4be1-93e0-eaaf4960ed20 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:42.144955 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82f6c1-98c3-4be1-93e0-eaaf4960ed20 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:42.146315 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 417/832] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:53:42.152484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce609ac-d034-465d-9cdc-b7bec690e8fb None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:42.152772 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce609ac-d034-465d-9cdc-b7bec690e8fb None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:42.153054 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce609ac-d034-465d-9cdc-b7bec690e8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:42.172291 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-5ce609ac-d034-465d-9cdc-b7bec690e8fb None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:42.174797 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-5ce609ac-d034-465d-9cdc-b7bec690e8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:42.192630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ee3a7-d5b3-4388-800f-5b99bc8e7fd7 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:42.207223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:42.209396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:42.230428 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f42dd-414c-49e1-a5fe-ad01c43baf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:42.231464 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f42dd-414c-49e1-a5fe-ad01c43baf1f None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:42.232074 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f42dd-414c-49e1-a5fe-ad01c43baf1f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:42.232421 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f42dd-414c-49e1-a5fe-ad01c43baf1f None admin] PATH_INFO: `/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/f88dfe620ff04ffe88088a8a23c7d6e9/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:42.234435 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f42dd-414c-49e1-a5fe-ad01c43baf1f None admin] RBAC: Authorizing `identity:create_grant(project_id=3f777cd6a0c744e594f8e29ab58cf8bd, user_id=f88dfe620ff04ffe88088a8a23c7d6e9, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:42.235983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f42dd-414c-49e1-a5fe-ad01c43baf1f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:42.262186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 418/833] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:53:42.289912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce609ac-d034-465d-9cdc-b7bec690e8fb None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:42.307933 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 416/834] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:53:42 2022] PUT /identity/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/f88dfe620ff04ffe88088a8a23c7d6e9/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:53:42.424600 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce609ac-d034-465d-9cdc-b7bec690e8fb None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:42.468192 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799e352-11d7-4ad2-82ea-ccefc8efd6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:42.469009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799e352-11d7-4ad2-82ea-ccefc8efd6dc None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:42.469360 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799e352-11d7-4ad2-82ea-ccefc8efd6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:42.469639 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799e352-11d7-4ad2-82ea-ccefc8efd6dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:42.482385 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799e352-11d7-4ad2-82ea-ccefc8efd6dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:42.492574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-9799e352-11d7-4ad2-82ea-ccefc8efd6dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:42.492574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:42.492574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:42.492574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:42.492574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:42.492574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:42.492574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:42.492574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:42.492574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:42.492574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:42.492574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:42.492574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:42.492574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:42.492574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:42.492574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:42.492574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:42.494048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:42.494048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:42.494048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:42.494048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:42.494048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:42.494048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:42.494048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:42.494048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:42.494048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:42.494048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:42.494048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:42.494048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:42.494048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:42.494048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:42.494048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:42.494048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:42.495067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:42.495067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:42.495067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:42.495067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:42.495067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:42.495067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:42.495067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:42.495067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:42.495067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:42.495067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:42.495067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:53:42.495067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:42.495067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 419/835] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:53:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:42.501039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f42dd-414c-49e1-a5fe-ad01c43baf1f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:42.621949 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f42dd-414c-49e1-a5fe-ad01c43baf1f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:42.659614 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25dc1bf-fa9b-4aab-a8d3-86c21010fbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:42.660292 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25dc1bf-fa9b-4aab-a8d3-86c21010fbe4 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:42.660572 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25dc1bf-fa9b-4aab-a8d3-86c21010fbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:42.660847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25dc1bf-fa9b-4aab-a8d3-86c21010fbe4 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:42.661599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25dc1bf-fa9b-4aab-a8d3-86c21010fbe4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:42.661901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25dc1bf-fa9b-4aab-a8d3-86c21010fbe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:42.663321 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25dc1bf-fa9b-4aab-a8d3-86c21010fbe4 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:42.673714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 417/836] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:53:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:42.679682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799e352-11d7-4ad2-82ea-ccefc8efd6dc None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:42.704215 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:42.796832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799e352-11d7-4ad2-82ea-ccefc8efd6dc None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:42.834602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0919a-3c2d-47c8-8c88-e76d1ddcc81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:42.835331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0919a-3c2d-47c8-8c88-e76d1ddcc81a None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:42.836210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0919a-3c2d-47c8-8c88-e76d1ddcc81a None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:42.836479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0919a-3c2d-47c8-8c88-e76d1ddcc81a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:42.852592 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0919a-3c2d-47c8-8c88-e76d1ddcc81a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:42.854135 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0919a-3c2d-47c8-8c88-e76d1ddcc81a None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:42.876173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-9fc0919a-3c2d-47c8-8c88-e76d1ddcc81a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:42.876173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:42.876173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:42.876173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:42.876173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:42.876173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:42.876173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:42.876173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:42.876173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:42.876173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:42.876173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:42.876173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:42.876173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:42.876173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:42.876173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:42.878290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:42.880604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:42.880604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:42.880604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:42.880604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:42.880604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 420/837] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:42.884175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25dc1bf-fa9b-4aab-a8d3-86c21010fbe4 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:42.929769 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:42.930113 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07981e26-f938-49cf-b5e8-50f8ac9edbd3 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:42.930585 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07981e26-f938-49cf-b5e8-50f8ac9edbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:42.931208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07981e26-f938-49cf-b5e8-50f8ac9edbd3 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:42.934208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 421/838] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:42.939642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcafc68-f834-48d0-b37b-277737fb18ed None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:42.939971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcafc68-f834-48d0-b37b-277737fb18ed None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:42.940428 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcafc68-f834-48d0-b37b-277737fb18ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:42.962253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-8dcafc68-f834-48d0-b37b-277737fb18ed None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:42.963917 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-8dcafc68-f834-48d0-b37b-277737fb18ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:43.024321 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25dc1bf-fa9b-4aab-a8d3-86c21010fbe4 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:43.075580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa397c-d1d5-4e6b-92ba-b43496a5dc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:43.075870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 422/839] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:53:43.076373 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa397c-d1d5-4e6b-92ba-b43496a5dc49 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:43.076660 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa397c-d1d5-4e6b-92ba-b43496a5dc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:43.076941 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa397c-d1d5-4e6b-92ba-b43496a5dc49 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:43.077684 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa397c-d1d5-4e6b-92ba-b43496a5dc49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:43.077994 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa397c-d1d5-4e6b-92ba-b43496a5dc49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:43.086600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa397c-d1d5-4e6b-92ba-b43496a5dc49 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:43.099435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcafc68-f834-48d0-b37b-277737fb18ed None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:43.100429 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 418/840] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:43.102570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:43.107318 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa397c-d1d5-4e6b-92ba-b43496a5dc49 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:43.215966 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:43.223165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa397c-d1d5-4e6b-92ba-b43496a5dc49 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:43.223375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcafc68-f834-48d0-b37b-277737fb18ed None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:43.263188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f27f6-8025-4ea7-be11-1cfe8e3a5066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:43.264118 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f27f6-8025-4ea7-be11-1cfe8e3a5066 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:43.264470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f27f6-8025-4ea7-be11-1cfe8e3a5066 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:43.264837 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f27f6-8025-4ea7-be11-1cfe8e3a5066 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:43.265197 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11f9c9-edb1-4c52-8168-a673a62275c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:43.266231 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11f9c9-edb1-4c52-8168-a673a62275c9 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:43.266744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11f9c9-edb1-4c52-8168-a673a62275c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:43.267318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11f9c9-edb1-4c52-8168-a673a62275c9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:43.268350 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11f9c9-edb1-4c52-8168-a673a62275c9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:43.269990 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11f9c9-edb1-4c52-8168-a673a62275c9 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:43.274408 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f27f6-8025-4ea7-be11-1cfe8e3a5066 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:43.275914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f27f6-8025-4ea7-be11-1cfe8e3a5066 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-7f11f9c9-edb1-4c52-8168-a673a62275c9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:43.285295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:43.291458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:43.291458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:43.291458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:43.291458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:43.291458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:43.291458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:43.291458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:53:43.291458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:53:43.291458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:53:43.291458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:53:43.291458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:43.291458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:43.291458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:43.291458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:43.291458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:43.291458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:43.292329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:43.292329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:43.292329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:43.292329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:43.292329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:43.292329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:43.292329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:43.292329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:43.292329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:43.292329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:43.292329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:53:43.292329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:43.292329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:53:43.292329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:43.292329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 423/841] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:53:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:43.293847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-120f27f6-8025-4ea7-be11-1cfe8e3a5066 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:53:43.293847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:43.293847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:43.293847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:43.293847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:43.293847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:43.293847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:43.293847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:43.293847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:43.293847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:43.293847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:43.293847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:43.293847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:43.293847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:43.293847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:43.293847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:43.294833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:43.295808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:43.295808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:43.295808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:43.295808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:43.295808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:43.295808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:43.295808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:43.295808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:43.295808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:53:43.295808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:43.297782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11f9c9-edb1-4c52-8168-a673a62275c9 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:43.298620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 419/842] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Jul 18 14:53:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:43.300623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f27f6-8025-4ea7-be11-1cfe8e3a5066 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:43.376431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:43.416315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f27f6-8025-4ea7-be11-1cfe8e3a5066 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:43.419762 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11f9c9-edb1-4c52-8168-a673a62275c9 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:43.458757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b0eea-d440-4340-b0c6-50fbfcedf578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:43.459445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0eea-d440-4340-b0c6-50fbfcedf578 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:43.459728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0eea-d440-4340-b0c6-50fbfcedf578 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:43.460015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0eea-d440-4340-b0c6-50fbfcedf578 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:43.461668 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b0eea-d440-4340-b0c6-50fbfcedf578 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:43.462177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b0eea-d440-4340-b0c6-50fbfcedf578 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:43.462975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d949529-cc45-49a1-bf0b-3568eecb5e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:43.463727 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b0eea-d440-4340-b0c6-50fbfcedf578 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:43.464120 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d949529-cc45-49a1-bf0b-3568eecb5e05 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:43.464652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d949529-cc45-49a1-bf0b-3568eecb5e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:43.465141 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d949529-cc45-49a1-bf0b-3568eecb5e05 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:43.466323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d949529-cc45-49a1-bf0b-3568eecb5e05 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:43.466960 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d949529-cc45-49a1-bf0b-3568eecb5e05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:43.468573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d949529-cc45-49a1-bf0b-3568eecb5e05 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:43.474754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 424/843] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:53:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:43.480867 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d949529-cc45-49a1-bf0b-3568eecb5e05 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:43.489190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 420/844] 10.222.0.83 () {56 vars in 1149 bytes} [Mon Jul 18 14:53:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:43.490323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b0eea-d440-4340-b0c6-50fbfcedf578 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:43.507003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:43.603537 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d949529-cc45-49a1-bf0b-3568eecb5e05 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:43.607348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b0eea-d440-4340-b0c6-50fbfcedf578 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:43.640875 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:43.647429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c0a6f-c651-4c30-9a8a-8524ad75a0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:43.648689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c0a6f-c651-4c30-9a8a-8524ad75a0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:43.649241 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c0a6f-c651-4c30-9a8a-8524ad75a0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:43.649729 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c0a6f-c651-4c30-9a8a-8524ad75a0c4 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:43.650658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c0a6f-c651-4c30-9a8a-8524ad75a0c4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:43.651051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c0a6f-c651-4c30-9a8a-8524ad75a0c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:43.651505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85103dcf-8457-4c8d-8798-484295da7745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:43.652395 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85103dcf-8457-4c8d-8798-484295da7745 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:43.652667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c0a6f-c651-4c30-9a8a-8524ad75a0c4 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:43.653137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85103dcf-8457-4c8d-8798-484295da7745 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:43.653582 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85103dcf-8457-4c8d-8798-484295da7745 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:43.666741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 425/845] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:53:43 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:43.742489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85103dcf-8457-4c8d-8798-484295da7745 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:43.744424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85103dcf-8457-4c8d-8798-484295da7745 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:43.762298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:43.877625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 421/846] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 14:53:43.883608 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c0a6f-c651-4c30-9a8a-8524ad75a0c4 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:44.000931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c0a6f-c651-4c30-9a8a-8524ad75a0c4 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:44.037836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6d45e-7c3b-4c0f-be4c-50241e0cb0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:44.038740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6d45e-7c3b-4c0f-be4c-50241e0cb0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:44.039032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6d45e-7c3b-4c0f-be4c-50241e0cb0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:44.039313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6d45e-7c3b-4c0f-be4c-50241e0cb0e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:44.051192 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6d45e-7c3b-4c0f-be4c-50241e0cb0e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:44.052671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6d45e-7c3b-4c0f-be4c-50241e0cb0e7 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:44.072049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-c8a6d45e-7c3b-4c0f-be4c-50241e0cb0e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:44.072049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:44.072049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:44.072049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:44.072049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:44.072049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:44.072049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:44.072049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:44.072049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:44.072049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:44.072049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:44.072049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:44.072049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:44.072049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:44.072049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:44.073059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:44.073975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:44.073975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:44.073975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:44.073975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:44.073975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 426/847] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:44.079082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85103dcf-8457-4c8d-8798-484295da7745 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:44.148979 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e1277-57d2-47e5-a11e-34ee02abb48f None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:44.149487 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e1277-57d2-47e5-a11e-34ee02abb48f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:44.149746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e1277-57d2-47e5-a11e-34ee02abb48f None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:44.154225 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 427/848] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:44.162567 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfe7be-e998-4f5e-b76c-0df737460bb1 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:44.162567 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfe7be-e998-4f5e-b76c-0df737460bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:44.162567 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfe7be-e998-4f5e-b76c-0df737460bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:44.185211 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d0dfe7be-e998-4f5e-b76c-0df737460bb1 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:44.186436 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d0dfe7be-e998-4f5e-b76c-0df737460bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:44.236598 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85103dcf-8457-4c8d-8798-484295da7745 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:44.251038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:44.257707 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:44.274835 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 428/849] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:53:44.283519 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e45b25-7363-4973-ba0f-3679d5358d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:44.285098 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e45b25-7363-4973-ba0f-3679d5358d42 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:44.285672 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e45b25-7363-4973-ba0f-3679d5358d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:44.286170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e45b25-7363-4973-ba0f-3679d5358d42 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:44.287446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e45b25-7363-4973-ba0f-3679d5358d42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:44.288328 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e45b25-7363-4973-ba0f-3679d5358d42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:44.291066 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dfe7be-e998-4f5e-b76c-0df737460bb1 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:44.291829 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e45b25-7363-4973-ba0f-3679d5358d42 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:44.306784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 422/850] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:44.312589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e45b25-7363-4973-ba0f-3679d5358d42 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:44.408686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dfe7be-e998-4f5e-b76c-0df737460bb1 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:44.449620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e45b25-7363-4973-ba0f-3679d5358d42 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:44.461623 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbc00c-574b-4c82-8e39-0f56b351f4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:44.464168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbc00c-574b-4c82-8e39-0f56b351f4da None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:44.464738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbc00c-574b-4c82-8e39-0f56b351f4da None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:44.465237 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbc00c-574b-4c82-8e39-0f56b351f4da None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:44.467027 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bbc00c-574b-4c82-8e39-0f56b351f4da None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:44.469000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bbc00c-574b-4c82-8e39-0f56b351f4da None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:44.487565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-b5bbc00c-574b-4c82-8e39-0f56b351f4da None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:53:44.487565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:44.487565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:53:44.487565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:53:44.487565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:53:44.487565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:53:44.487565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:53:44.487565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:44.487565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:53:44.487565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:44.487565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:44.487565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:44.487565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:44.487565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:44.487565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:44.487565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:44.488673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:44.488673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:44.488673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:44.488673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:44.488673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:44.488673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:44.488673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:44.488673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:53:44.488673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:53:44.488673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:53:44.488673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:53:44.488673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:44.488673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:44.488673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:44.488673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:44.488673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:44.489540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:44.489540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:44.489540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:44.489540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:44.489540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:44.489540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:44.489540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:44.489540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:44.489540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:44.489540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:44.489540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:44.489540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:53:44.489540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:44.489540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:53:44.489540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:44.498268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 429/851] 10.222.0.83 () {56 vars in 1132 bytes} [Mon Jul 18 14:53:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:44.498526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebd3b7-60ac-4eff-97aa-ab0cb5020583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:44.498526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebd3b7-60ac-4eff-97aa-ab0cb5020583 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:44.498526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebd3b7-60ac-4eff-97aa-ab0cb5020583 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:44.498526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebd3b7-60ac-4eff-97aa-ab0cb5020583 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:44.502110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbc00c-574b-4c82-8e39-0f56b351f4da None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:44.506234 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebd3b7-60ac-4eff-97aa-ab0cb5020583 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:44.508686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebd3b7-60ac-4eff-97aa-ab0cb5020583 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:44.529634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-43ebd3b7-60ac-4eff-97aa-ab0cb5020583 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:44.529634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:44.529634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:44.529634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:44.529634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:44.529634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:44.529634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:44.529634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:44.529634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:44.529634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:44.529634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:44.529634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:44.529634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:44.529634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:44.529634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:44.530706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:44.531621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:44.531621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:44.531621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:44.531621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:44.531621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:44.531621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:44.531621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:44.531621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:44.533956 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 423/852] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:44.540030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebd3b7-60ac-4eff-97aa-ab0cb5020583 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:44.574940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:44.582964 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:44.650008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbc00c-574b-4c82-8e39-0f56b351f4da None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:44.662450 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebd3b7-60ac-4eff-97aa-ab0cb5020583 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:44.687333 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f28b27-caee-47cb-b645-7e449cc01c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:44.688134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f28b27-caee-47cb-b645-7e449cc01c72 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:44.688425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f28b27-caee-47cb-b645-7e449cc01c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:44.688707 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f28b27-caee-47cb-b645-7e449cc01c72 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:44.689427 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f28b27-caee-47cb-b645-7e449cc01c72 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:44.689739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f28b27-caee-47cb-b645-7e449cc01c72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:44.691171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f28b27-caee-47cb-b645-7e449cc01c72 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:44.698059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 430/853] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Jul 18 14:53:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:44.699910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2cf82-473e-49cb-a6e9-d01a8ef78564 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:44.700312 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2cf82-473e-49cb-a6e9-d01a8ef78564 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:44.700672 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2cf82-473e-49cb-a6e9-d01a8ef78564 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:44.706412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 431/854] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:44.706412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2cf82-473e-49cb-a6e9-d01a8ef78564 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:44.712067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1866c-d30a-49bc-a8b3-ffcfaff4ccd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:44.713109 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1866c-d30a-49bc-a8b3-ffcfaff4ccd6 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:44.713109 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1866c-d30a-49bc-a8b3-ffcfaff4ccd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:44.713849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1866c-d30a-49bc-a8b3-ffcfaff4ccd6 None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:44.715873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1866c-d30a-49bc-a8b3-ffcfaff4ccd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:44.715873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1866c-d30a-49bc-a8b3-ffcfaff4ccd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:44.716109 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1866c-d30a-49bc-a8b3-ffcfaff4ccd6 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:44.734300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 424/855] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:44.734300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272099d-3ab2-46e6-bddc-b0685abdb143 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:44.734300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272099d-3ab2-46e6-bddc-b0685abdb143 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:44.734300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272099d-3ab2-46e6-bddc-b0685abdb143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:44.756843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-0272099d-3ab2-46e6-bddc-b0685abdb143 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:44.759299 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-0272099d-3ab2-46e6-bddc-b0685abdb143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:44.850565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 425/856] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:44.850565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272099d-3ab2-46e6-bddc-b0685abdb143 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:44.851544 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2cf82-473e-49cb-a6e9-d01a8ef78564 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:44.852449 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:44.887277 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:44.891376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c9bc0-5e20-4954-b4ea-7ff0adff9f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:44.892068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c9bc0-5e20-4954-b4ea-7ff0adff9f03 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:44.892342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c9bc0-5e20-4954-b4ea-7ff0adff9f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:44.892622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c9bc0-5e20-4954-b4ea-7ff0adff9f03 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:44.893347 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c9bc0-5e20-4954-b4ea-7ff0adff9f03 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:44.893768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c9bc0-5e20-4954-b4ea-7ff0adff9f03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:44.895107 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c9bc0-5e20-4954-b4ea-7ff0adff9f03 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:44.901263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 432/857] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Jul 18 14:53:44 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:44.902308 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c9bc0-5e20-4954-b4ea-7ff0adff9f03 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:45.011721 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272099d-3ab2-46e6-bddc-b0685abdb143 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:45.047617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c9bc0-5e20-4954-b4ea-7ff0adff9f03 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:45.065622 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceee817-c090-4227-ae6b-be0e04957f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:45.066743 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceee817-c090-4227-ae6b-be0e04957f1c None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:45.067173 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceee817-c090-4227-ae6b-be0e04957f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:45.067525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceee817-c090-4227-ae6b-be0e04957f1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:45.069174 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceee817-c090-4227-ae6b-be0e04957f1c None admin] RBAC: Adding query filter params (role.id=29b8b950372848ffa9a75e778ee66b4c, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=692fef632836474da09f0e8a009ffa4e) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:45.070117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceee817-c090-4227-ae6b-be0e04957f1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:45.073419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceee817-c090-4227-ae6b-be0e04957f1c None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:45.086605 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 426/858] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:44 2022] GET /identity/v3/role_assignments?user.id=692fef632836474da09f0e8a009ffa4e&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=29b8b950372848ffa9a75e778ee66b4c => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:45.119335 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65aa23-b990-4c1d-869d-6aa206950ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:45.120169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65aa23-b990-4c1d-869d-6aa206950ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:45.120460 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65aa23-b990-4c1d-869d-6aa206950ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:45.120744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65aa23-b990-4c1d-869d-6aa206950ae4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:45.135460 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65aa23-b990-4c1d-869d-6aa206950ae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:45.143824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-7f65aa23-b990-4c1d-869d-6aa206950ae4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:53:45.143824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:45.143824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:45.143824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:45.143824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:45.143824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:45.143824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:45.143824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:45.143824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:45.143824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:45.143824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:45.143824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:45.143824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:45.143824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:45.143824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:45.143824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:45.144911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:45.144911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:45.144911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:45.144911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:45.144911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:45.144911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:45.144911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:45.144911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:45.144911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:45.144911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:45.144911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:45.144911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:45.144911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:45.144911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:45.144911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:45.144911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:45.145777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:45.145777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:45.145777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:45.145777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:45.145777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:45.145777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:45.145777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:45.145777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:45.145777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:45.145777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:45.145777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:53:45.145777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:45.145777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 433/859] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:53:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:45.153486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceee817-c090-4227-ae6b-be0e04957f1c None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:45.282599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:45.299004 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceee817-c090-4227-ae6b-be0e04957f1c None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:45.344689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef1af1-f448-4019-af52-6daa0966350d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:45.345247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef1af1-f448-4019-af52-6daa0966350d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:45.345538 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef1af1-f448-4019-af52-6daa0966350d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:45.345815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef1af1-f448-4019-af52-6daa0966350d None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:45.346612 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef1af1-f448-4019-af52-6daa0966350d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:45.346929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef1af1-f448-4019-af52-6daa0966350d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:45.348339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef1af1-f448-4019-af52-6daa0966350d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:45.359137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 427/860] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:53:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:45.365007 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65aa23-b990-4c1d-869d-6aa206950ae4 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:45.398451 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:45.484764 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65aa23-b990-4c1d-869d-6aa206950ae4 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:45.529716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a9cf5-8a83-4522-b57f-e6c26a720f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:45.530631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a9cf5-8a83-4522-b57f-e6c26a720f9e None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:45.530921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a9cf5-8a83-4522-b57f-e6c26a720f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:45.531201 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a9cf5-8a83-4522-b57f-e6c26a720f9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:45.544455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a9cf5-8a83-4522-b57f-e6c26a720f9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:45.545861 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a9cf5-8a83-4522-b57f-e6c26a720f9e None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:45.567334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-254a9cf5-8a83-4522-b57f-e6c26a720f9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:45.567334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:45.567334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:45.567334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:45.567334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:45.567334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:45.567334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:45.567334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:45.567334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:45.567334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:45.567334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:45.567334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:45.567334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:45.567334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:45.567334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:45.568340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:45.569261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:45.569261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:45.569261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:45.569261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:45.569261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 434/861] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:45.577135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef1af1-f448-4019-af52-6daa0966350d None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:45.705867 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef1af1-f448-4019-af52-6daa0966350d None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:45.771504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af95eff-7a18-4b7b-aa9d-dc9f3ac63856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:45.772689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af95eff-7a18-4b7b-aa9d-dc9f3ac63856 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:45.773202 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af95eff-7a18-4b7b-aa9d-dc9f3ac63856 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:45.773713 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af95eff-7a18-4b7b-aa9d-dc9f3ac63856 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:45.775106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af95eff-7a18-4b7b-aa9d-dc9f3ac63856 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:45.775655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af95eff-7a18-4b7b-aa9d-dc9f3ac63856 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:45.778306 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af95eff-7a18-4b7b-aa9d-dc9f3ac63856 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:45.810822 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 428/862] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:45.812848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a9cf5-8a83-4522-b57f-e6c26a720f9e None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:45.906921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:45.923887 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a9cf5-8a83-4522-b57f-e6c26a720f9e None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:45.962163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155db046-7a00-4583-82ac-ccaa67089cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:45.963424 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155db046-7a00-4583-82ac-ccaa67089cbe None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:45.964063 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155db046-7a00-4583-82ac-ccaa67089cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:45.964353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155db046-7a00-4583-82ac-ccaa67089cbe None admin] PATH_INFO: `/v3/users/swift` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:45.972641 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155db046-7a00-4583-82ac-ccaa67089cbe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:45.974132 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155db046-7a00-4583-82ac-ccaa67089cbe None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:45.990671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-155db046-7a00-4583-82ac-ccaa67089cbe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:53:45.990671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:45.990671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:45.990671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:45.990671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:45.990671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:45.990671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:45.990671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:45.990671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:45.990671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:45.990671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:45.990671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:45.990671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:45.990671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:45.990671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:45.990671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:45.994369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:45.995384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:45.995384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:45.995384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:45.995384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:45.995384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:45.995384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:45.995384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:45.995384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:45.995384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:53:45.995384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:45.995384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 435/863] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:53:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:46.002088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af95eff-7a18-4b7b-aa9d-dc9f3ac63856 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:46.021694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:46.027541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04b728-dc2c-457b-927c-74fa5f1ffb4e None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:46.027882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04b728-dc2c-457b-927c-74fa5f1ffb4e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:46.028138 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04b728-dc2c-457b-927c-74fa5f1ffb4e None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:46.029761 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 436/864] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:46.037574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cf8b8-92c8-4834-be97-0cddc201c9b2 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:46.037940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cf8b8-92c8-4834-be97-0cddc201c9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:46.039239 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cf8b8-92c8-4834-be97-0cddc201c9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:46.058272 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-be5cf8b8-92c8-4834-be97-0cddc201c9b2 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:46.059705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-be5cf8b8-92c8-4834-be97-0cddc201c9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:46.136680 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 437/865] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:53:46.180120 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cf8b8-92c8-4834-be97-0cddc201c9b2 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:46.190959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af95eff-7a18-4b7b-aa9d-dc9f3ac63856 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:46.252310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33aca8d-bf0a-4360-a7bc-46125df61fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:46.252310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33aca8d-bf0a-4360-a7bc-46125df61fee None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:46.253105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33aca8d-bf0a-4360-a7bc-46125df61fee None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:46.253438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33aca8d-bf0a-4360-a7bc-46125df61fee None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:46.254365 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33aca8d-bf0a-4360-a7bc-46125df61fee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:46.254783 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33aca8d-bf0a-4360-a7bc-46125df61fee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:46.263087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33aca8d-bf0a-4360-a7bc-46125df61fee None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:46.280825 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 429/866] 10.222.0.83 () {56 vars in 1135 bytes} [Mon Jul 18 14:53:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:46.291791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33aca8d-bf0a-4360-a7bc-46125df61fee None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:46.311967 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:46.348005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cf8b8-92c8-4834-be97-0cddc201c9b2 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:46.369068 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:46.389484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce164b6d-5362-4e9d-8281-eb1d68cf4680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:46.390127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce164b6d-5362-4e9d-8281-eb1d68cf4680 None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:46.390355 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce164b6d-5362-4e9d-8281-eb1d68cf4680 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:46.390665 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce164b6d-5362-4e9d-8281-eb1d68cf4680 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:46.391418 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce164b6d-5362-4e9d-8281-eb1d68cf4680 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:46.392865 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce164b6d-5362-4e9d-8281-eb1d68cf4680 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:46.407838 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33aca8d-bf0a-4360-a7bc-46125df61fee None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:46.445682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd855c85-980e-4e41-9b15-e075da8bf5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:46.446338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd855c85-980e-4e41-9b15-e075da8bf5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:46.446659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd855c85-980e-4e41-9b15-e075da8bf5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:46.446944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd855c85-980e-4e41-9b15-e075da8bf5d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:46.470136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 438/867] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Jul 18 14:53:46 2022] POST /identity/v3/services => generated 251 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:53:46.536596 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd855c85-980e-4e41-9b15-e075da8bf5d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:46.538305 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd855c85-980e-4e41-9b15-e075da8bf5d7 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:46.619170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:46.660740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 430/868] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:46 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:53:46.667132 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce164b6d-5362-4e9d-8281-eb1d68cf4680 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:46.799224 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce164b6d-5362-4e9d-8281-eb1d68cf4680 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:46.829018 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:46.838735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3601d8-9c23-48ba-a9ca-4dce61f13b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:46.839553 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3601d8-9c23-48ba-a9ca-4dce61f13b48 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:46.839842 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3601d8-9c23-48ba-a9ca-4dce61f13b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:46.840125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3601d8-9c23-48ba-a9ca-4dce61f13b48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:46.852261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3601d8-9c23-48ba-a9ca-4dce61f13b48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:46.853857 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3601d8-9c23-48ba-a9ca-4dce61f13b48 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:46.873705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-aa3601d8-9c23-48ba-a9ca-4dce61f13b48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:46.873705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:46.873705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:46.873705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:46.873705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:46.873705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:46.873705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:46.873705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:46.873705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:46.873705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:46.873705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:46.873705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:46.873705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:46.873705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:46.873705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:46.874757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:46.876150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:46.876150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:46.876150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:46.876150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:46.876150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 439/869] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:46.881239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd855c85-980e-4e41-9b15-e075da8bf5d7 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:46.997408 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd855c85-980e-4e41-9b15-e075da8bf5d7 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:47.036507 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc61cd-d4c4-4a0b-9bb8-2c0b2646481d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:47.037686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc61cd-d4c4-4a0b-9bb8-2c0b2646481d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:47.038231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc61cd-d4c4-4a0b-9bb8-2c0b2646481d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:47.038773 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc61cd-d4c4-4a0b-9bb8-2c0b2646481d None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:47.039747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc61cd-d4c4-4a0b-9bb8-2c0b2646481d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:47.040280 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc61cd-d4c4-4a0b-9bb8-2c0b2646481d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:47.042139 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc61cd-d4c4-4a0b-9bb8-2c0b2646481d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:47.058501 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 431/870] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:47.070335 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3601d8-9c23-48ba-a9ca-4dce61f13b48 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:47.189656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3601d8-9c23-48ba-a9ca-4dce61f13b48 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:47.226088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7158f8-429b-4193-a421-01897f9a46c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:47.226962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7158f8-429b-4193-a421-01897f9a46c4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:47.227252 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7158f8-429b-4193-a421-01897f9a46c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:47.227531 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7158f8-429b-4193-a421-01897f9a46c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:47.235336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7158f8-429b-4193-a421-01897f9a46c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:47.236830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7158f8-429b-4193-a421-01897f9a46c4 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:47.252617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-1e7158f8-429b-4193-a421-01897f9a46c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:47.252617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:47.252617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:47.252617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:47.252617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:47.252617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:47.252617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:47.252617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:47.252617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:47.252617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:47.252617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:47.252617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:47.252617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:47.252617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:47.252617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:47.253636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:47.260001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:47.260001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:47.260001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:47.260001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:47.260001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:47.260001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:47.260001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:47.260001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:47.260001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 440/871] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:47.262821 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc61cd-d4c4-4a0b-9bb8-2c0b2646481d None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:47.270047 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d6f49-c3de-46f5-9996-98a279bd1ca8 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:47.270417 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d6f49-c3de-46f5-9996-98a279bd1ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:47.270663 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d6f49-c3de-46f5-9996-98a279bd1ca8 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:47.272386 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 441/872] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:53:47.278120 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49e8bc-d2e6-47c3-9222-4c6028d72eda None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:47.278420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49e8bc-d2e6-47c3-9222-4c6028d72eda None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:47.278715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49e8bc-d2e6-47c3-9222-4c6028d72eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:47.300893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-6a49e8bc-d2e6-47c3-9222-4c6028d72eda None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:47.302118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6a49e8bc-d2e6-47c3-9222-4c6028d72eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:47.303611 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:47.335118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:47.376599 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 442/873] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:47 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:53:47.393499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc61cd-d4c4-4a0b-9bb8-2c0b2646481d None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:47.408580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49e8bc-d2e6-47c3-9222-4c6028d72eda None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:47.436566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca82e76-8ab9-4c1e-b5bc-e089fe17ddb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:47.437513 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca82e76-8ab9-4c1e-b5bc-e089fe17ddb0 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:47.437898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca82e76-8ab9-4c1e-b5bc-e089fe17ddb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:47.438273 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca82e76-8ab9-4c1e-b5bc-e089fe17ddb0 None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:47.439196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca82e76-8ab9-4c1e-b5bc-e089fe17ddb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:47.439604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca82e76-8ab9-4c1e-b5bc-e089fe17ddb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:47.441413 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca82e76-8ab9-4c1e-b5bc-e089fe17ddb0 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:47.455459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 432/874] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:47.461459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca82e76-8ab9-4c1e-b5bc-e089fe17ddb0 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:47.526321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49e8bc-d2e6-47c3-9222-4c6028d72eda None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:47.567806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:47.577125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119abf3-d1ff-43db-8fc5-d794beb5ad76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:47.578275 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119abf3-d1ff-43db-8fc5-d794beb5ad76 None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:47.579102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119abf3-d1ff-43db-8fc5-d794beb5ad76 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:47.579836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119abf3-d1ff-43db-8fc5-d794beb5ad76 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:47.585955 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8119abf3-d1ff-43db-8fc5-d794beb5ad76 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:47.585955 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8119abf3-d1ff-43db-8fc5-d794beb5ad76 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:47.586189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca82e76-8ab9-4c1e-b5bc-e089fe17ddb0 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:47.626194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a724300-6a44-4d4d-88ee-3586b54b7638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:47.627103 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a724300-6a44-4d4d-88ee-3586b54b7638 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:47.627446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a724300-6a44-4d4d-88ee-3586b54b7638 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:47.627721 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a724300-6a44-4d4d-88ee-3586b54b7638 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:47.628560 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a724300-6a44-4d4d-88ee-3586b54b7638 None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=f88dfe620ff04ffe88088a8a23c7d6e9) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:47.628871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a724300-6a44-4d4d-88ee-3586b54b7638 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:47.630409 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a724300-6a44-4d4d-88ee-3586b54b7638 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:47.642637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 433/875] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:47 2022] GET /identity/v3/role_assignments?user.id=f88dfe620ff04ffe88088a8a23c7d6e9&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=5891920310b54787bd1152d736d2a3f1 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:47.648836 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8e329-3502-4a04-96d3-8454d2b210cf None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:47.649192 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8e329-3502-4a04-96d3-8454d2b210cf None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:47.649472 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8e329-3502-4a04-96d3-8454d2b210cf None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:47.651022 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 434/876] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:53:47.657354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49f0a2-e93e-47c8-8123-811f2eca4b6a None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:47.657712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49f0a2-e93e-47c8-8123-811f2eca4b6a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:47.658037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49f0a2-e93e-47c8-8123-811f2eca4b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:47.676682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-7a49f0a2-e93e-47c8-8123-811f2eca4b6a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:47.678021 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7a49f0a2-e93e-47c8-8123-811f2eca4b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:47.691322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 443/877] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Jul 18 14:53:47 2022] POST /identity/v3/services => generated 269 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:53:47.788663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 435/878] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:53:47.815258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119abf3-d1ff-43db-8fc5-d794beb5ad76 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:47.851107 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:47.959097 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119abf3-d1ff-43db-8fc5-d794beb5ad76 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:47.999595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1856b62-ca4c-48cc-9063-2f3cd6fa74d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:48.000510 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1856b62-ca4c-48cc-9063-2f3cd6fa74d5 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:48.000809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1856b62-ca4c-48cc-9063-2f3cd6fa74d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:48.001082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1856b62-ca4c-48cc-9063-2f3cd6fa74d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:48.013414 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1856b62-ca4c-48cc-9063-2f3cd6fa74d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:48.021952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-a1856b62-ca4c-48cc-9063-2f3cd6fa74d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:53:48.021952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:48.021952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:48.021952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:48.021952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:48.021952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:48.021952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:48.021952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:48.021952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:48.021952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:48.021952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:48.021952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:48.021952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:48.021952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:48.021952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:48.021952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:48.023688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:48.023688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:48.023688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:48.023688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:48.023688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:48.023688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:48.023688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:48.023688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:48.023688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:48.023688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:48.023688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:48.023688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:48.023688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:48.023688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:48.023688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:48.023688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:48.024581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:48.024581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:48.024581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:48.024581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:48.024581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:48.024581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:48.024581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:48.024581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:48.024581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:48.024581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:48.024581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:53:48.024581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:48.024581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 444/879] 10.222.0.83 () {56 vars in 1120 bytes} [Mon Jul 18 14:53:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:48.030324 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49f0a2-e93e-47c8-8123-811f2eca4b6a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:48.120384 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:48.145427 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49f0a2-e93e-47c8-8123-811f2eca4b6a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:48.185056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38b836-75a2-4caf-b372-b670d7c0eccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:48.186159 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38b836-75a2-4caf-b372-b670d7c0eccd None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:48.186740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38b836-75a2-4caf-b372-b670d7c0eccd None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:48.187552 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38b836-75a2-4caf-b372-b670d7c0eccd None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:48.189168 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38b836-75a2-4caf-b372-b670d7c0eccd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:48.189987 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38b836-75a2-4caf-b372-b670d7c0eccd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:48.191959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38b836-75a2-4caf-b372-b670d7c0eccd None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:48.202919 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 436/880] 10.222.0.83 () {56 vars in 1115 bytes} [Mon Jul 18 14:53:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:48.210591 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1856b62-ca4c-48cc-9063-2f3cd6fa74d5 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:48.310982 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:48.324698 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1856b62-ca4c-48cc-9063-2f3cd6fa74d5 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:48.361748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe7049-1924-4736-8402-4e35f68d5262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:48.362580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe7049-1924-4736-8402-4e35f68d5262 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:48.362872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe7049-1924-4736-8402-4e35f68d5262 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:48.363152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe7049-1924-4736-8402-4e35f68d5262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:48.375325 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe7049-1924-4736-8402-4e35f68d5262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:48.376730 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe7049-1924-4736-8402-4e35f68d5262 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:48.396082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-e3fe7049-1924-4736-8402-4e35f68d5262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:48.396082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:48.396082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:48.396082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:48.396082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:48.396082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:48.396082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:48.396082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:48.396082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:48.396082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:48.396082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:48.396082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:48.396082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:48.396082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:48.396082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:48.397068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:48.398220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:48.398220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:48.398220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:48.398220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:48.398220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 445/881] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:48.404981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38b836-75a2-4caf-b372-b670d7c0eccd None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:48.554719 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38b836-75a2-4caf-b372-b670d7c0eccd None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:48.594159 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36934487-490e-4334-974f-a1a6fc013bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:48.595359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36934487-490e-4334-974f-a1a6fc013bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:48.596008 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36934487-490e-4334-974f-a1a6fc013bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:48.596524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36934487-490e-4334-974f-a1a6fc013bd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:48.597556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36934487-490e-4334-974f-a1a6fc013bd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:48.598090 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36934487-490e-4334-974f-a1a6fc013bd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:48.600080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36934487-490e-4334-974f-a1a6fc013bd3 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:48.609240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:48.617843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 437/882] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:48.623204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe7049-1924-4736-8402-4e35f68d5262 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:48.733011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe7049-1924-4736-8402-4e35f68d5262 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:48.769460 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898ca52-ad81-4c0b-8ae3-d51104ee5cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:48.770296 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898ca52-ad81-4c0b-8ae3-d51104ee5cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:48.770611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898ca52-ad81-4c0b-8ae3-d51104ee5cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:48.770890 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898ca52-ad81-4c0b-8ae3-d51104ee5cf5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:48.778993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898ca52-ad81-4c0b-8ae3-d51104ee5cf5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:48.780487 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898ca52-ad81-4c0b-8ae3-d51104ee5cf5 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:48.787509 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:48.795762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5023c-b42b-4532-b8ab-82a5b612e12a None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:48.796716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5023c-b42b-4532-b8ab-82a5b612e12a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:48.801625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5023c-b42b-4532-b8ab-82a5b612e12a None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:48.801789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-a898ca52-ad81-4c0b-8ae3-d51104ee5cf5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:53:48.801789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:48.801789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:48.801789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:48.801789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:48.801789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:48.801789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:48.801789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:48.801789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:48.801789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:48.801789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:48.801789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:48.801789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:48.801789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:48.801789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:48.801789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:48.802721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:48.804616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:48.804616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:48.804616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:48.804616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:48.804616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:48.804616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:48.804616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:48.804616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:48.804616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:53:48.804616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:48.804616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 446/883] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Jul 18 14:53:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:48.805244 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 438/884] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:48.806577 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68d94e-aa76-4435-a8ba-da5ebebd0de2 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:48.807071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68d94e-aa76-4435-a8ba-da5ebebd0de2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:48.807537 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68d94e-aa76-4435-a8ba-da5ebebd0de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:48.813685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5023c-b42b-4532-b8ab-82a5b612e12a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:48.827693 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-1e68d94e-aa76-4435-a8ba-da5ebebd0de2 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:48.829324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-1e68d94e-aa76-4435-a8ba-da5ebebd0de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:48.922567 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 447/885] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:48.940643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68d94e-aa76-4435-a8ba-da5ebebd0de2 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:48.965809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5023c-b42b-4532-b8ab-82a5b612e12a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:49.021359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92813796-aa99-49d1-a681-c72fc8b32b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:49.021950 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92813796-aa99-49d1-a681-c72fc8b32b18 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:49.022244 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92813796-aa99-49d1-a681-c72fc8b32b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:49.022573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92813796-aa99-49d1-a681-c72fc8b32b18 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:49.023746 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92813796-aa99-49d1-a681-c72fc8b32b18 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:49.024081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92813796-aa99-49d1-a681-c72fc8b32b18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:49.026623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92813796-aa99-49d1-a681-c72fc8b32b18 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:49.054674 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 439/886] 10.222.0.83 () {56 vars in 1149 bytes} [Mon Jul 18 14:53:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:49.061335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92813796-aa99-49d1-a681-c72fc8b32b18 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:49.085839 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68d94e-aa76-4435-a8ba-da5ebebd0de2 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:49.128550 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:49.134278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:49.138621 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139622a8-d0e3-44d9-8f67-47982077ba58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:49.139811 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139622a8-d0e3-44d9-8f67-47982077ba58 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:49.140183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139622a8-d0e3-44d9-8f67-47982077ba58 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:49.140541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139622a8-d0e3-44d9-8f67-47982077ba58 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:49.141453 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139622a8-d0e3-44d9-8f67-47982077ba58 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:49.143574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139622a8-d0e3-44d9-8f67-47982077ba58 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-139622a8-d0e3-44d9-8f67-47982077ba58 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:49.163166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:49.164480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:49.164480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:49.164480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:49.164480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:49.164480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:49.164480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:49.164480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:53:49.164480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:53:49.164480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:53:49.164480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:53:49.164480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:49.164480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:49.164480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:49.164480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:49.164480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:49.164480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:49.165541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:49.165541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:49.165541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:49.165541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:49.165541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:49.165541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:49.165541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:49.165541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:49.165541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:49.165541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:49.165541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:53:49.165541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:49.165541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:53:49.165541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:49.173193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 448/887] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:53:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:49.173452 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139622a8-d0e3-44d9-8f67-47982077ba58 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:49.194366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92813796-aa99-49d1-a681-c72fc8b32b18 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:49.243366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad4588-8bb2-4a86-9fb6-cc8fb3890242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:49.244578 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad4588-8bb2-4a86-9fb6-cc8fb3890242 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:49.244946 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad4588-8bb2-4a86-9fb6-cc8fb3890242 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:49.245298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad4588-8bb2-4a86-9fb6-cc8fb3890242 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:49.326168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139622a8-d0e3-44d9-8f67-47982077ba58 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:49.334082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad4588-8bb2-4a86-9fb6-cc8fb3890242 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:49.336015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad4588-8bb2-4a86-9fb6-cc8fb3890242 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:49.373718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247ae1c-dc4c-40c0-a8ad-1dceb4dd630c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:49.374404 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247ae1c-dc4c-40c0-a8ad-1dceb4dd630c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:49.374722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247ae1c-dc4c-40c0-a8ad-1dceb4dd630c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:49.375007 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247ae1c-dc4c-40c0-a8ad-1dceb4dd630c None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:49.375735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247ae1c-dc4c-40c0-a8ad-1dceb4dd630c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:49.376047 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247ae1c-dc4c-40c0-a8ad-1dceb4dd630c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:49.377744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247ae1c-dc4c-40c0-a8ad-1dceb4dd630c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:49.389139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 449/888] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:53:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:49.391208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247ae1c-dc4c-40c0-a8ad-1dceb4dd630c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:49.409653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:49.478573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 440/889] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:49 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:53:49.481632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad4588-8bb2-4a86-9fb6-cc8fb3890242 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:49.490122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:49.545624 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247ae1c-dc4c-40c0-a8ad-1dceb4dd630c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:49.589496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36b574-af36-4d60-9a4d-58d294c75d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:49.589992 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36b574-af36-4d60-9a4d-58d294c75d8f None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:49.590280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36b574-af36-4d60-9a4d-58d294c75d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:49.590576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36b574-af36-4d60-9a4d-58d294c75d8f None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:49.591316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36b574-af36-4d60-9a4d-58d294c75d8f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:49.591626 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36b574-af36-4d60-9a4d-58d294c75d8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:49.593604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36b574-af36-4d60-9a4d-58d294c75d8f None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:49.601328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 450/890] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:53:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:49.608287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36b574-af36-4d60-9a4d-58d294c75d8f None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:49.628788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad4588-8bb2-4a86-9fb6-cc8fb3890242 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:49.693470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a3354-884e-4522-9c0c-d64407c5f149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:49.695397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a3354-884e-4522-9c0c-d64407c5f149 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:49.696712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a3354-884e-4522-9c0c-d64407c5f149 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:49.697162 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a3354-884e-4522-9c0c-d64407c5f149 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:49.713821 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a3354-884e-4522-9c0c-d64407c5f149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:49.715440 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a3354-884e-4522-9c0c-d64407c5f149 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:49.740678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-254a3354-884e-4522-9c0c-d64407c5f149 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:49.740678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:49.740678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:49.740678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:49.740678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:49.740678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:49.740678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:49.740678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:49.740678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:49.740678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:49.740678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:49.740678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:49.740678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:49.740678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:49.740678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:49.742243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:49.743178 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:49.743178 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:49.743178 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:49.743178 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:49.743403 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36b574-af36-4d60-9a4d-58d294c75d8f None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:49.747425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 441/891] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:49.749552 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a3354-884e-4522-9c0c-d64407c5f149 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:49.765515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:49.777041 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:49.794020 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19376f3e-b753-430b-b180-742d38593b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:49.794862 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19376f3e-b753-430b-b180-742d38593b0f None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:49.795225 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19376f3e-b753-430b-b180-742d38593b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:49.795604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19376f3e-b753-430b-b180-742d38593b0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:49.796534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19376f3e-b753-430b-b180-742d38593b0f None admin] RBAC: Adding query filter params (interface=public, service_id=594daacaf7b7401e95351f753146cf81, region_id=RegionOne) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:49.796973 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19376f3e-b753-430b-b180-742d38593b0f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:49.799539 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19376f3e-b753-430b-b180-742d38593b0f None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:49.815226 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 451/892] 10.222.0.83 () {56 vars in 1251 bytes} [Mon Jul 18 14:53:49 2022] GET /identity/v3/endpoints?service_id=594daacaf7b7401e95351f753146cf81&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:49.816286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19376f3e-b753-430b-b180-742d38593b0f None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:49.940576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a3354-884e-4522-9c0c-d64407c5f149 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:49.954938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19376f3e-b753-430b-b180-742d38593b0f None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:49.986886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de0caa-3171-459d-b5bf-9d09bb182343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:49.987728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de0caa-3171-459d-b5bf-9d09bb182343 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:49.988020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de0caa-3171-459d-b5bf-9d09bb182343 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:49.988580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de0caa-3171-459d-b5bf-9d09bb182343 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:49.989324 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de0caa-3171-459d-b5bf-9d09bb182343 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:49.989748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de0caa-3171-459d-b5bf-9d09bb182343 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:49.991211 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de0caa-3171-459d-b5bf-9d09bb182343 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:50.000809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122f749-2983-4cc8-b1b6-8dd2d381340b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:50.001502 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122f749-2983-4cc8-b1b6-8dd2d381340b None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:50.001791 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122f749-2983-4cc8-b1b6-8dd2d381340b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:50.002071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122f749-2983-4cc8-b1b6-8dd2d381340b None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:50.002811 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122f749-2983-4cc8-b1b6-8dd2d381340b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:50.003873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 442/893] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:50.004210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122f749-2983-4cc8-b1b6-8dd2d381340b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:50.005213 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25fe8c-94ac-48ef-9d6f-330c20d0e231 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:50.005500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25fe8c-94ac-48ef-9d6f-330c20d0e231 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:50.005780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25fe8c-94ac-48ef-9d6f-330c20d0e231 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:50.007078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 443/894] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:50.008163 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6250c27-439e-4e4c-a78c-6c6802f81a7a None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:50.008447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6250c27-439e-4e4c-a78c-6c6802f81a7a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:50.008725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6250c27-439e-4e4c-a78c-6c6802f81a7a None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:50.010134 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 444/895] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:50.013154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd815fb6-9c8d-4c04-8387-52a706a510f4 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:50.013554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd815fb6-9c8d-4c04-8387-52a706a510f4 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:50.013929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd815fb6-9c8d-4c04-8387-52a706a510f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:50.014445 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 452/896] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Jul 18 14:53:49 2022] GET /identity/v3/services => generated 1268 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:53:50.015584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122f749-2983-4cc8-b1b6-8dd2d381340b None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:50.037632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-fd815fb6-9c8d-4c04-8387-52a706a510f4 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:50.039657 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-fd815fb6-9c8d-4c04-8387-52a706a510f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:50.085091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:50.144465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122f749-2983-4cc8-b1b6-8dd2d381340b None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:50.152565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:50.159359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 445/897] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:50.168241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121832fb-7aee-44a1-932e-c0833f07889d None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:50.168667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121832fb-7aee-44a1-932e-c0833f07889d None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:50.168816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121832fb-7aee-44a1-932e-c0833f07889d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:50.191596 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-121832fb-7aee-44a1-932e-c0833f07889d None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:50.192344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-121832fb-7aee-44a1-932e-c0833f07889d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:50.197513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a559ea-ac05-473c-8f69-fba7a8bf209c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:50.198362 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a559ea-ac05-473c-8f69-fba7a8bf209c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:50.199237 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a559ea-ac05-473c-8f69-fba7a8bf209c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:50.199596 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a559ea-ac05-473c-8f69-fba7a8bf209c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:50.208057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a559ea-ac05-473c-8f69-fba7a8bf209c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:50.209529 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a559ea-ac05-473c-8f69-fba7a8bf209c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:50.225457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-63a559ea-ac05-473c-8f69-fba7a8bf209c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:50.225457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:50.225457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:50.225457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:50.225457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:50.225457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:50.225457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:50.225457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:50.225457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:50.225457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:50.225457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:50.225457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:50.225457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:50.225457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:50.225457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:50.226581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:50.227565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:50.227565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:50.227565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:50.227565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:50.227565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:50.227565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:50.227565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:50.227565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:50.227565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 453/898] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:50.228137 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a559ea-ac05-473c-8f69-fba7a8bf209c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:50.281159 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 446/899] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:50.282449 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121832fb-7aee-44a1-932e-c0833f07889d None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:50.363667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a559ea-ac05-473c-8f69-fba7a8bf209c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:50.377501 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:50.414626 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121832fb-7aee-44a1-932e-c0833f07889d None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:50.418254 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479b317-a457-44e9-b2cf-927a5dd037d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:50.419381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479b317-a457-44e9-b2cf-927a5dd037d7 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:50.419843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479b317-a457-44e9-b2cf-927a5dd037d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:50.420308 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479b317-a457-44e9-b2cf-927a5dd037d7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:50.421484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479b317-a457-44e9-b2cf-927a5dd037d7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:50.423954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479b317-a457-44e9-b2cf-927a5dd037d7 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-2479b317-a457-44e9-b2cf-927a5dd037d7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:50.447723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:50.448874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:50.448874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:50.448874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:50.448874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:50.448874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:50.448874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:50.448874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:53:50.448874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:53:50.448874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:53:50.448874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:53:50.448874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:50.448874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:50.448874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:50.448874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:50.448874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:50.448874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:50.449784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:50.449784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:50.449784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:50.449784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:50.449784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:50.449784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:50.449784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:50.449784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:50.449784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:50.449784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:50.449784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:53:50.449784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:50.449784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:53:50.449784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:50.449784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 454/900] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Jul 18 14:53:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:50.457265 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479b317-a457-44e9-b2cf-927a5dd037d7 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:50.465276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:50.475949 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fa1d0-c5bc-400c-8c0b-d045aa012f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:50.480734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fa1d0-c5bc-400c-8c0b-d045aa012f48 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:50.480734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fa1d0-c5bc-400c-8c0b-d045aa012f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:50.480734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fa1d0-c5bc-400c-8c0b-d045aa012f48 None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:50.480734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fa1d0-c5bc-400c-8c0b-d045aa012f48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:50.480734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fa1d0-c5bc-400c-8c0b-d045aa012f48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:50.480734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fa1d0-c5bc-400c-8c0b-d045aa012f48 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:50.498914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 447/901] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:50.498914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fa1d0-c5bc-400c-8c0b-d045aa012f48 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:50.590785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479b317-a457-44e9-b2cf-927a5dd037d7 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:50.621508 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fa1d0-c5bc-400c-8c0b-d045aa012f48 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:50.627177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83758a9c-b126-4f03-84f9-49f7f3959ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:50.628051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83758a9c-b126-4f03-84f9-49f7f3959ffd None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:50.628326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83758a9c-b126-4f03-84f9-49f7f3959ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:50.628602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83758a9c-b126-4f03-84f9-49f7f3959ffd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:50.629255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83758a9c-b126-4f03-84f9-49f7f3959ffd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:50.630655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83758a9c-b126-4f03-84f9-49f7f3959ffd None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:50.644538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-83758a9c-b126-4f03-84f9-49f7f3959ffd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:53:50.644538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:50.644538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:53:50.644538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:53:50.644538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:53:50.644538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:53:50.644538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:53:50.644538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:50.644538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:53:50.644538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:50.644538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:50.644538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:50.644538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:50.644538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:50.644538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:50.644538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:50.645614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:50.645614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:50.645614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:50.645614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:50.645614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:50.645614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:50.645614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:50.645614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:53:50.645614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:53:50.645614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:53:50.645614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:53:50.645614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:50.645614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:50.645614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:50.645614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:50.645614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:50.646503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:50.646503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:50.646503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:50.646503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:50.646503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:50.646503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:50.646503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:50.646503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:50.646503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:50.646503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:50.646503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:50.646503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:53:50.646503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:50.646503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:53:50.646503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:50.646503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 455/902] 10.222.0.83 () {56 vars in 1132 bytes} [Mon Jul 18 14:53:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:50.648207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83758a9c-b126-4f03-84f9-49f7f3959ffd None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:50.666830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a421ce-993c-4c14-bfa2-7243244f5c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:50.667499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a421ce-993c-4c14-bfa2-7243244f5c10 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:50.667784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a421ce-993c-4c14-bfa2-7243244f5c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:50.668064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a421ce-993c-4c14-bfa2-7243244f5c10 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:50.668780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a421ce-993c-4c14-bfa2-7243244f5c10 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:50.669074 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a421ce-993c-4c14-bfa2-7243244f5c10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:50.670466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a421ce-993c-4c14-bfa2-7243244f5c10 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:50.680278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 448/903] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Jul 18 14:53:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:50.680278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a421ce-993c-4c14-bfa2-7243244f5c10 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:50.700550 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:50.736647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:50.777916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83758a9c-b126-4f03-84f9-49f7f3959ffd None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:50.801522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a421ce-993c-4c14-bfa2-7243244f5c10 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:50.825332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39587f35-8bc3-4dee-8f11-edf915e9573c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:50.826005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39587f35-8bc3-4dee-8f11-edf915e9573c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:50.826297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39587f35-8bc3-4dee-8f11-edf915e9573c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:50.826605 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39587f35-8bc3-4dee-8f11-edf915e9573c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:50.827450 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39587f35-8bc3-4dee-8f11-edf915e9573c None admin] RBAC: Adding query filter params (role.id=f7a8b3159cc4475ebac07df3543883cc, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=692fef632836474da09f0e8a009ffa4e) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:50.827841 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39587f35-8bc3-4dee-8f11-edf915e9573c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:50.829274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39587f35-8bc3-4dee-8f11-edf915e9573c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:50.836118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 456/904] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:53:50 2022] GET /identity/v3/role_assignments?user.id=692fef632836474da09f0e8a009ffa4e&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=f7a8b3159cc4475ebac07df3543883cc => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:50.836962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39587f35-8bc3-4dee-8f11-edf915e9573c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:50.840378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd786a2-d88e-466f-a76a-03e2fe443a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:50.841455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd786a2-d88e-466f-a76a-03e2fe443a78 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:50.841968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd786a2-d88e-466f-a76a-03e2fe443a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:50.842457 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd786a2-d88e-466f-a76a-03e2fe443a78 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:50.843459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd786a2-d88e-466f-a76a-03e2fe443a78 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:50.843996 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd786a2-d88e-466f-a76a-03e2fe443a78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:50.845702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd786a2-d88e-466f-a76a-03e2fe443a78 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:50.852974 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 449/905] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Jul 18 14:53:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:50.858988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd786a2-d88e-466f-a76a-03e2fe443a78 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:50.991254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd786a2-d88e-466f-a76a-03e2fe443a78 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:51.011726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:51.018724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39587f35-8bc3-4dee-8f11-edf915e9573c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:51.021157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:51.038033 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421ba03-54b3-4129-bc23-d09f7de5c2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:51.039116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421ba03-54b3-4129-bc23-d09f7de5c2bb None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:51.039482 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421ba03-54b3-4129-bc23-d09f7de5c2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:51.039806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421ba03-54b3-4129-bc23-d09f7de5c2bb None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:51.040639 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421ba03-54b3-4129-bc23-d09f7de5c2bb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:51.040956 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421ba03-54b3-4129-bc23-d09f7de5c2bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:51.042364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421ba03-54b3-4129-bc23-d09f7de5c2bb None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:51.048439 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 450/906] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Jul 18 14:53:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:51.059777 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421ba03-54b3-4129-bc23-d09f7de5c2bb None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:51.063865 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9d897-52b6-4663-99d5-2b03cc75f354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:51.065036 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9d897-52b6-4663-99d5-2b03cc75f354 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:51.065478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9d897-52b6-4663-99d5-2b03cc75f354 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:51.065907 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9d897-52b6-4663-99d5-2b03cc75f354 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:51.066820 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9d897-52b6-4663-99d5-2b03cc75f354 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:51.067300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9d897-52b6-4663-99d5-2b03cc75f354 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:51.069382 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9d897-52b6-4663-99d5-2b03cc75f354 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:51.075419 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 457/907] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Jul 18 14:53:50 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:51.206071 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421ba03-54b3-4129-bc23-d09f7de5c2bb None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:51.247520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6cdac-9a7d-4c82-bb98-c1bfb196a3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:51.248131 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6cdac-9a7d-4c82-bb98-c1bfb196a3cf None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:51.248421 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6cdac-9a7d-4c82-bb98-c1bfb196a3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:51.248701 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6cdac-9a7d-4c82-bb98-c1bfb196a3cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:51.249545 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6cdac-9a7d-4c82-bb98-c1bfb196a3cf None admin] RBAC: Adding query filter params (interface=public, service_id=518f09fd993d4f76aef211af4680430d, region_id=RegionOne) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:51.249873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6cdac-9a7d-4c82-bb98-c1bfb196a3cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:51.251444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6cdac-9a7d-4c82-bb98-c1bfb196a3cf None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:51.261547 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 451/908] 10.222.0.83 () {56 vars in 1251 bytes} [Mon Jul 18 14:53:51 2022] GET /identity/v3/endpoints?service_id=518f09fd993d4f76aef211af4680430d&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:51.266939 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9d897-52b6-4663-99d5-2b03cc75f354 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:51.384858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9d897-52b6-4663-99d5-2b03cc75f354 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:51.421744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4007318-22f7-4c04-a0e2-722d2c951668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:51.422881 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4007318-22f7-4c04-a0e2-722d2c951668 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:51.423386 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4007318-22f7-4c04-a0e2-722d2c951668 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:51.423875 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4007318-22f7-4c04-a0e2-722d2c951668 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:51.424776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4007318-22f7-4c04-a0e2-722d2c951668 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:51.426500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4007318-22f7-4c04-a0e2-722d2c951668 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:51.434519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 458/909] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Jul 18 14:53:51 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:53:52.244358 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d83f2a-8e80-4d7a-98a0-1996351a885a None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:52.244909 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d83f2a-8e80-4d7a-98a0-1996351a885a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:52.244909 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d83f2a-8e80-4d7a-98a0-1996351a885a None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:52.246434 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 452/910] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:52.252000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090018ec-3fc5-4b92-bb51-84837cae9043 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:52.252479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090018ec-3fc5-4b92-bb51-84837cae9043 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:52.252558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090018ec-3fc5-4b92-bb51-84837cae9043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:52.277646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-090018ec-3fc5-4b92-bb51-84837cae9043 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:52.279778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-090018ec-3fc5-4b92-bb51-84837cae9043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:52.323573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:52.373787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 459/911] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:52.397514 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d83f2a-8e80-4d7a-98a0-1996351a885a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:52.450348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:52.509564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d83f2a-8e80-4d7a-98a0-1996351a885a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:52.545305 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353396e6-62f7-46ae-807d-28d08fbf3d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:52.546148 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353396e6-62f7-46ae-807d-28d08fbf3d2b None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:52.546455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353396e6-62f7-46ae-807d-28d08fbf3d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:52.546756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353396e6-62f7-46ae-807d-28d08fbf3d2b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:52.547403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353396e6-62f7-46ae-807d-28d08fbf3d2b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:52.548846 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353396e6-62f7-46ae-807d-28d08fbf3d2b None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-353396e6-62f7-46ae-807d-28d08fbf3d2b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:52.561250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:52.562347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:52.562347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:52.562347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:52.562347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:52.562347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:52.562347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:52.562347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:53:52.562347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:53:52.562347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:53:52.562347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:53:52.562347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:52.562347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:52.562347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:52.562347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:52.562347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:52.562347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:52.563703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:52.563703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:52.563703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:52.563703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:52.563703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:52.563703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:52.563703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:52.563703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:52.563703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:52.563703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:52.563703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:53:52.563703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:52.563703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:53:52.563703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:52.563703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 453/912] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:53:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:52.569693 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090018ec-3fc5-4b92-bb51-84837cae9043 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:52.684922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090018ec-3fc5-4b92-bb51-84837cae9043 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:52.723332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527ec07-6db6-40a3-8068-bd5ef36729f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:52.724017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527ec07-6db6-40a3-8068-bd5ef36729f1 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:52.724327 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527ec07-6db6-40a3-8068-bd5ef36729f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:52.724591 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527ec07-6db6-40a3-8068-bd5ef36729f1 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:52.725812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527ec07-6db6-40a3-8068-bd5ef36729f1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:52.726123 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527ec07-6db6-40a3-8068-bd5ef36729f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:52.727573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527ec07-6db6-40a3-8068-bd5ef36729f1 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:52.733656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 460/913] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:53:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:52.739353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353396e6-62f7-46ae-807d-28d08fbf3d2b None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:52.899140 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353396e6-62f7-46ae-807d-28d08fbf3d2b None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:52.909202 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:52.936793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77a00e-0dcc-4985-a871-7330a5f7e48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:52.937472 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77a00e-0dcc-4985-a871-7330a5f7e48d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:52.937761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77a00e-0dcc-4985-a871-7330a5f7e48d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:52.938044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77a00e-0dcc-4985-a871-7330a5f7e48d None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:52.938793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77a00e-0dcc-4985-a871-7330a5f7e48d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:52.939105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77a00e-0dcc-4985-a871-7330a5f7e48d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:52.940506 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77a00e-0dcc-4985-a871-7330a5f7e48d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:52.947021 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 454/914] 10.222.0.83 () {56 vars in 1114 bytes} [Mon Jul 18 14:53:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:52.956139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527ec07-6db6-40a3-8068-bd5ef36729f1 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:53.005552 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:53.101964 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527ec07-6db6-40a3-8068-bd5ef36729f1 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:53.140567 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14817d10-e991-4294-965f-622206c80f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:53.141243 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14817d10-e991-4294-965f-622206c80f14 None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:53.141534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14817d10-e991-4294-965f-622206c80f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:53.141824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14817d10-e991-4294-965f-622206c80f14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:53.142623 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14817d10-e991-4294-965f-622206c80f14 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:53.144023 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14817d10-e991-4294-965f-622206c80f14 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:53.255496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 461/915] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Jul 18 14:53:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:53:53.261855 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1842093-e000-4012-b8f0-9237d881922b None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:53.261855 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1842093-e000-4012-b8f0-9237d881922b None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:53.261855 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1842093-e000-4012-b8f0-9237d881922b None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:53.263057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 455/916] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:53.268614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12fd4e-0abc-403c-b6d0-99c03f9fcc1c None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:53.268917 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12fd4e-0abc-403c-b6d0-99c03f9fcc1c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:53.269299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12fd4e-0abc-403c-b6d0-99c03f9fcc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:53.295843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-1d12fd4e-0abc-403c-b6d0-99c03f9fcc1c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:53.297163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-1d12fd4e-0abc-403c-b6d0-99c03f9fcc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:53.397301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 462/917] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:53.423333 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1842093-e000-4012-b8f0-9237d881922b None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:53.455967 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdf685-6fcb-4000-a2a8-f0df3cf7e12e None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:53.456355 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdf685-6fcb-4000-a2a8-f0df3cf7e12e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:53.456532 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdf685-6fcb-4000-a2a8-f0df3cf7e12e None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:53.458355 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 463/918] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:53:53.464316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b8182-7e3c-4b51-8f04-158e5e7e339e None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:53.464993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b8182-7e3c-4b51-8f04-158e5e7e339e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:53.465276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b8182-7e3c-4b51-8f04-158e5e7e339e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:53.478703 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:53.486510 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-e78b8182-7e3c-4b51-8f04-158e5e7e339e None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:53.488204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e78b8182-7e3c-4b51-8f04-158e5e7e339e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:53.579326 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1842093-e000-4012-b8f0-9237d881922b None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:53.587178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 464/919] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:53.614715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b8182-7e3c-4b51-8f04-158e5e7e339e None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:53.621699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653f877-f0d1-4ae2-93b2-38f076558451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:53.622863 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653f877-f0d1-4ae2-93b2-38f076558451 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:53.623322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653f877-f0d1-4ae2-93b2-38f076558451 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:53.623815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653f877-f0d1-4ae2-93b2-38f076558451 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:53.640536 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653f877-f0d1-4ae2-93b2-38f076558451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:53.653315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-7653f877-f0d1-4ae2-93b2-38f076558451 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:53:53.653315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:53.653315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:53.653315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:53.653315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:53.653315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:53.653315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:53.653315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:53.653315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:53.653315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:53.653315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:53.653315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:53.653315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:53.653315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:53.653315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:53.653315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:53.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:53.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:53.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:53.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:53.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:53.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:53.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:53.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:53.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:53.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:53.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:53.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:53.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:53.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:53.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:53.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:53.655502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:53.655502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:53.655502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:53.655502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:53.655502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:53.655502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:53.655502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:53.655502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:53.655502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:53.655502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:53.655502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:53:53.655502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:53.656976 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 456/920] 10.222.0.83 () {56 vars in 1120 bytes} [Mon Jul 18 14:53:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:53.663320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653f877-f0d1-4ae2-93b2-38f076558451 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:53.706913 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:53.750836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b8182-7e3c-4b51-8f04-158e5e7e339e None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:53.801036 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045ebef-12e2-4606-b36f-b127e485bf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:53.801714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045ebef-12e2-4606-b36f-b127e485bf9f None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:53.801998 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045ebef-12e2-4606-b36f-b127e485bf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:53.802274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045ebef-12e2-4606-b36f-b127e485bf9f None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:53.803576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045ebef-12e2-4606-b36f-b127e485bf9f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:53.805082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045ebef-12e2-4606-b36f-b127e485bf9f None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:53.808308 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:53.813744 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653f877-f0d1-4ae2-93b2-38f076558451 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:53.854830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5106bc3-0d3b-443c-964a-7b31551fe365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:53.855427 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5106bc3-0d3b-443c-964a-7b31551fe365 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:53.855714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5106bc3-0d3b-443c-964a-7b31551fe365 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:53.856307 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5106bc3-0d3b-443c-964a-7b31551fe365 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:53.857049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5106bc3-0d3b-443c-964a-7b31551fe365 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:53.857351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5106bc3-0d3b-443c-964a-7b31551fe365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:53.858765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5106bc3-0d3b-443c-964a-7b31551fe365 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:53.868532 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 457/921] 10.222.0.83 () {56 vars in 1115 bytes} [Mon Jul 18 14:53:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:53.869879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87faf853-f7a6-4a0f-883c-a276d6a32312 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:53.870167 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87faf853-f7a6-4a0f-883c-a276d6a32312 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:53.870444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87faf853-f7a6-4a0f-883c-a276d6a32312 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:53.871881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 458/922] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:53.876793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87faf853-f7a6-4a0f-883c-a276d6a32312 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:53.905977 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 465/923] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Jul 18 14:53:53 2022] POST /identity/v3/services => generated 241 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:53:53.912351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9eb3d9-534c-4a97-8da8-e8c1708f8d86 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:53.912827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9eb3d9-534c-4a97-8da8-e8c1708f8d86 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:53.913203 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9eb3d9-534c-4a97-8da8-e8c1708f8d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:53.938269 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-fb9eb3d9-534c-4a97-8da8-e8c1708f8d86 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:53.940663 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-fb9eb3d9-534c-4a97-8da8-e8c1708f8d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:54.030464 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:54.059872 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87faf853-f7a6-4a0f-883c-a276d6a32312 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:54.070729 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 466/924] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:54.099157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9eb3d9-534c-4a97-8da8-e8c1708f8d86 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:54.121114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44773b33-0328-4d69-b300-c137e7c79c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:54.122610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44773b33-0328-4d69-b300-c137e7c79c5c None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:54.122912 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44773b33-0328-4d69-b300-c137e7c79c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:54.123562 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44773b33-0328-4d69-b300-c137e7c79c5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:54.138525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44773b33-0328-4d69-b300-c137e7c79c5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:54.140669 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44773b33-0328-4d69-b300-c137e7c79c5c None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:54.161194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-44773b33-0328-4d69-b300-c137e7c79c5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:54.161194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:54.161194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:54.161194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:54.161194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:54.161194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:54.161194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:54.161194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:54.161194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:54.161194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:54.161194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:54.161194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:54.161194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:54.161194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:54.161194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:54.162223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:54.163752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:54.163752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:54.163752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:54.163752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:54.163752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 459/925] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:54.169695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44773b33-0328-4d69-b300-c137e7c79c5c None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:54.233846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9eb3d9-534c-4a97-8da8-e8c1708f8d86 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:54.267174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:54.283780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38965017-892e-4ba3-b5a1-25e1d592d096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:54.285218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38965017-892e-4ba3-b5a1-25e1d592d096 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:54.285674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38965017-892e-4ba3-b5a1-25e1d592d096 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:54.286016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38965017-892e-4ba3-b5a1-25e1d592d096 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:54.287386 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38965017-892e-4ba3-b5a1-25e1d592d096 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:54.289088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38965017-892e-4ba3-b5a1-25e1d592d096 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:54.295971 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44773b33-0328-4d69-b300-c137e7c79c5c None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:54.305189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-38965017-892e-4ba3-b5a1-25e1d592d096 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:53:54.305189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:54.305189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:53:54.305189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:53:54.305189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:53:54.305189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:53:54.305189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:53:54.305189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:54.305189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:53:54.305189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:54.305189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:54.305189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:54.305189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:54.305189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:54.305189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:54.305189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:54.306236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:54.306236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:54.306236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:54.306236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:54.306236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:54.306236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:54.306236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:54.306236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:53:54.306236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:53:54.306236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:53:54.306236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:53:54.306236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:54.306236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:54.306236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:54.306236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:54.306236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:54.307146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:54.307146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:54.307146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:54.307146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:54.307146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:54.307146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:54.307146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:54.307146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:54.307146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:54.307146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:54.307146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:54.307146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:53:54.307146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:54.307146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:53:54.307146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:54.312216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 467/926] 10.222.0.83 () {56 vars in 1132 bytes} [Mon Jul 18 14:53:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:54.313966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38965017-892e-4ba3-b5a1-25e1d592d096 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:54.353855 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2406e-54e0-4a06-bad4-19c371aa39e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:54.354594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2406e-54e0-4a06-bad4-19c371aa39e3 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:54.354872 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2406e-54e0-4a06-bad4-19c371aa39e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:54.355133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2406e-54e0-4a06-bad4-19c371aa39e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:54.355867 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2406e-54e0-4a06-bad4-19c371aa39e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:54.356432 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2406e-54e0-4a06-bad4-19c371aa39e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:54.357917 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2406e-54e0-4a06-bad4-19c371aa39e3 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:54.373616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 460/927] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:54.379906 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2406e-54e0-4a06-bad4-19c371aa39e3 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:54.384089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:54.452474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38965017-892e-4ba3-b5a1-25e1d592d096 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:54.500183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e95b9-177d-4c1f-b7fd-2ee5f5c9f5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:54.501563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e95b9-177d-4c1f-b7fd-2ee5f5c9f5bd None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:54.502197 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e95b9-177d-4c1f-b7fd-2ee5f5c9f5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:54.503466 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e95b9-177d-4c1f-b7fd-2ee5f5c9f5bd None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:54.504574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e95b9-177d-4c1f-b7fd-2ee5f5c9f5bd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:54.505723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e95b9-177d-4c1f-b7fd-2ee5f5c9f5bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:54.510083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e95b9-177d-4c1f-b7fd-2ee5f5c9f5bd None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:54.522662 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 468/928] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Jul 18 14:53:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:54.524676 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e95b9-177d-4c1f-b7fd-2ee5f5c9f5bd None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:54.535438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2406e-54e0-4a06-bad4-19c371aa39e3 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:54.582664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfbada-78b2-4033-a4ab-8c5afbdb2d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:54.583534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfbada-78b2-4033-a4ab-8c5afbdb2d45 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:54.583821 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfbada-78b2-4033-a4ab-8c5afbdb2d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:54.584098 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfbada-78b2-4033-a4ab-8c5afbdb2d45 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:54.594946 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:54.599058 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdfbada-78b2-4033-a4ab-8c5afbdb2d45 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:54.600704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdfbada-78b2-4033-a4ab-8c5afbdb2d45 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:54.622921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-abdfbada-78b2-4033-a4ab-8c5afbdb2d45 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:53:54.622921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:54.622921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:54.622921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:54.622921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:54.622921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:54.622921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:54.622921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:54.622921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:54.622921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:54.622921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:54.622921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:54.622921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:54.622921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:54.622921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:54.622921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:54.624081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:54.625096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:54.625096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:54.625096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:54.625096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:54.625096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:54.625096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:54.625096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:54.625096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:54.625096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:53:54.625096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:54.625096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 461/929] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Jul 18 14:53:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:54.637571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfbada-78b2-4033-a4ab-8c5afbdb2d45 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:54.665351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e95b9-177d-4c1f-b7fd-2ee5f5c9f5bd None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:54.719953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:54.734365 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519d78e-0247-49ff-b9f6-0e2c1087132f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:54.735110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519d78e-0247-49ff-b9f6-0e2c1087132f None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:54.735397 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519d78e-0247-49ff-b9f6-0e2c1087132f None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:54.735678 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519d78e-0247-49ff-b9f6-0e2c1087132f None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:54.736467 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519d78e-0247-49ff-b9f6-0e2c1087132f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:54.736783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519d78e-0247-49ff-b9f6-0e2c1087132f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:54.738284 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519d78e-0247-49ff-b9f6-0e2c1087132f None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:54.744748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 469/930] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Jul 18 14:53:54 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:54.750328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519d78e-0247-49ff-b9f6-0e2c1087132f None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:54.774041 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfbada-78b2-4033-a4ab-8c5afbdb2d45 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:54.824668 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762af8d-cae8-458a-8d33-bd3b50708610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:54.825356 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762af8d-cae8-458a-8d33-bd3b50708610 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:54.825642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762af8d-cae8-458a-8d33-bd3b50708610 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:54.825924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762af8d-cae8-458a-8d33-bd3b50708610 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:54.826930 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762af8d-cae8-458a-8d33-bd3b50708610 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:54.827241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762af8d-cae8-458a-8d33-bd3b50708610 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:54.828819 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762af8d-cae8-458a-8d33-bd3b50708610 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:54.856207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 462/931] 10.222.0.83 () {56 vars in 1149 bytes} [Mon Jul 18 14:53:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:54.857580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762af8d-cae8-458a-8d33-bd3b50708610 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:54.894103 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519d78e-0247-49ff-b9f6-0e2c1087132f None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:54.921161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:54.962614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531a9e1-2a32-46a6-b2e1-6c74f8a05a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:54.963653 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531a9e1-2a32-46a6-b2e1-6c74f8a05a3d None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:54.964098 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531a9e1-2a32-46a6-b2e1-6c74f8a05a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:54.964632 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531a9e1-2a32-46a6-b2e1-6c74f8a05a3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:54.965721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531a9e1-2a32-46a6-b2e1-6c74f8a05a3d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:54.967481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531a9e1-2a32-46a6-b2e1-6c74f8a05a3d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:54.989534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762af8d-cae8-458a-8d33-bd3b50708610 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:55.011943 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:55.027148 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44fa7eb-1308-4e60-b514-534aaaa7fc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:55.027947 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44fa7eb-1308-4e60-b514-534aaaa7fc2b None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:55.028229 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44fa7eb-1308-4e60-b514-534aaaa7fc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:55.028504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44fa7eb-1308-4e60-b514-534aaaa7fc2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:55.040125 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44fa7eb-1308-4e60-b514-534aaaa7fc2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:55.041750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44fa7eb-1308-4e60-b514-534aaaa7fc2b None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:55.061181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-c44fa7eb-1308-4e60-b514-534aaaa7fc2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:55.061181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:55.061181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:55.061181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:55.061181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:55.061181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:55.061181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:55.061181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:55.061181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:55.061181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:55.061181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:55.061181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:55.061181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:55.061181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:55.061181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:55.062580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:55.063551 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:55.063551 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:55.063551 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:55.063551 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:55.068216 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 463/932] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:55.069291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44fa7eb-1308-4e60-b514-534aaaa7fc2b None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:55.074918 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 470/933] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Jul 18 14:53:54 2022] POST /identity/v3/endpoints => generated 347 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:53:55.205536 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44fa7eb-1308-4e60-b514-534aaaa7fc2b None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:55.257865 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a8009-2f12-423f-a4d4-3824623f626c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:55.258947 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a8009-2f12-423f-a4d4-3824623f626c None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:55.259417 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a8009-2f12-423f-a4d4-3824623f626c None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:55.259948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a8009-2f12-423f-a4d4-3824623f626c None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:55.264056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a8009-2f12-423f-a4d4-3824623f626c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:55.264056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a8009-2f12-423f-a4d4-3824623f626c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:55.264056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a8009-2f12-423f-a4d4-3824623f626c None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:55.278727 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 464/934] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:55.281331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531a9e1-2a32-46a6-b2e1-6c74f8a05a3d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:55.399302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531a9e1-2a32-46a6-b2e1-6c74f8a05a3d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:55.441792 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a98bd7-8e02-46c5-95f5-c6a6aea68416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:55.442701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a98bd7-8e02-46c5-95f5-c6a6aea68416 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:55.443031 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a98bd7-8e02-46c5-95f5-c6a6aea68416 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:55.443314 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a98bd7-8e02-46c5-95f5-c6a6aea68416 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:55.452285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a98bd7-8e02-46c5-95f5-c6a6aea68416 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:55.453756 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a98bd7-8e02-46c5-95f5-c6a6aea68416 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:55.469927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-68a98bd7-8e02-46c5-95f5-c6a6aea68416 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:55.469927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:55.469927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:55.469927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:55.469927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:55.469927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:55.469927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:55.469927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:55.469927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:55.469927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:55.469927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:55.469927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:55.469927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:55.469927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:55.469927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:55.470910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:55.471847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:55.471847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:55.471847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:55.471847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:55.471847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:55.471847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:55.471847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:55.471847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:55.471847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 471/935] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:55.477095 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a8009-2f12-423f-a4d4-3824623f626c None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:55.501276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a250571-8749-4311-8a83-5cfaaa2d8d21 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:55.501675 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a250571-8749-4311-8a83-5cfaaa2d8d21 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:55.501883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a250571-8749-4311-8a83-5cfaaa2d8d21 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:55.506674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 472/936] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:53:55.509598 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a2766-6307-4e04-8648-0d05f40fac73 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:55.509896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a2766-6307-4e04-8648-0d05f40fac73 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:55.510185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a2766-6307-4e04-8648-0d05f40fac73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:55.515500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:55.530694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d40a2766-6307-4e04-8648-0d05f40fac73 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:55.532152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d40a2766-6307-4e04-8648-0d05f40fac73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:55.564145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:55.619335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a8009-2f12-423f-a4d4-3824623f626c None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:55.622814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 473/937] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:55.640152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a2766-6307-4e04-8648-0d05f40fac73 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:55.659539 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bd287-156f-49ab-ae4a-a0fc08f2e3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:55.660686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bd287-156f-49ab-ae4a-a0fc08f2e3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:55.661187 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bd287-156f-49ab-ae4a-a0fc08f2e3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:55.661700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bd287-156f-49ab-ae4a-a0fc08f2e3a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:55.663256 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bd287-156f-49ab-ae4a-a0fc08f2e3a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:55.663891 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bd287-156f-49ab-ae4a-a0fc08f2e3a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:55.665580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bd287-156f-49ab-ae4a-a0fc08f2e3a5 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:55.681530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 465/938] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:53:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:55.690577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bd287-156f-49ab-ae4a-a0fc08f2e3a5 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:55.754357 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a2766-6307-4e04-8648-0d05f40fac73 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:55.799547 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a7192-0c7d-4f12-b437-c04fbca515a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:55.800536 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a7192-0c7d-4f12-b437-c04fbca515a2 None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:55.800833 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a7192-0c7d-4f12-b437-c04fbca515a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:55.801121 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a7192-0c7d-4f12-b437-c04fbca515a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:55.802136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a7192-0c7d-4f12-b437-c04fbca515a2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:55.804998 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a7192-0c7d-4f12-b437-c04fbca515a2 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:55.807251 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: WARNING keystone.server.flask.common [None req-ca0a7192-0c7d-4f12-b437-c04fbca515a2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 14:53:55.810665 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:55.825779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bd287-156f-49ab-ae4a-a0fc08f2e3a5 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:55.867036 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b2282-f89a-4bd4-b76e-f67b85e0fc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:55.867951 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b2282-f89a-4bd4-b76e-f67b85e0fc98 None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:55.868239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b2282-f89a-4bd4-b76e-f67b85e0fc98 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:55.868521 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b2282-f89a-4bd4-b76e-f67b85e0fc98 None admin] PATH_INFO: `/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/692fef632836474da09f0e8a009ffa4e/roles/f7a8b3159cc4475ebac07df3543883cc` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:55.870987 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b2282-f89a-4bd4-b76e-f67b85e0fc98 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f777cd6a0c744e594f8e29ab58cf8bd, user_id=692fef632836474da09f0e8a009ffa4e, role_id=f7a8b3159cc4475ebac07df3543883cc)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:55.872473 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b2282-f89a-4bd4-b76e-f67b85e0fc98 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:55.881972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.sql.core [None req-ca0a7192-0c7d-4f12-b437-c04fbca515a2 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 14:53:55.881972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 14:53:55.881972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [parameters: {'id': 'cafe1992d7f34dc1a4791633686a6544', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 14:53:55.881972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80304) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 14:53:55.889850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-ca0a7192-0c7d-4f12-b437-c04fbca515a2 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cafe1992d7f34dc1a4791633686a6544.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cafe1992d7f34dc1a4791633686a6544. Jul 18 14:53:55.889850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:55.889850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 14:53:55.889850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 14:53:55.889850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 14:53:55.889850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 14:53:55.889850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 14:53:55.889850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 14:53:55.889850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 14:53:55.889850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application conn.query(q) Jul 18 14:53:55.889850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 14:53:55.889850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 14:53:55.889850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application result.read() Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 18 14:53:55.893040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application next(self.gen) Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application yield resource Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application next(self.gen) Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise self.value Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application session.commit() Jul 18 14:53:55.894709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self.session.flush() Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application compat.raise_( Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 14:53:55.897595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 14:53:55.900871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 14:53:55.900871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 14:53:55.900871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 14:53:55.900871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 14:53:55.900871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 14:53:55.900871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 14:53:55.900871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 14:53:55.900871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 14:53:55.900871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 14:53:55.900871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 14:53:55.900871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 14:53:55.900871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 14:53:55.900871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 14:53:55.900871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 14:53:55.900871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application conn.query(q) Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application result.read() Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 14:53:55.905713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 14:53:55.908253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 14:53:55.908253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 14:53:55.908253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 14:53:55.908253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 14:53:55.908253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 14:53:55.908253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 14:53:55.908253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 14:53:55.908253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 14:53:55.908253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [parameters: {'id': 'cafe1992d7f34dc1a4791633686a6544', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 14:53:55.908253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 14:53:55.908253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:55.908253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:55.908253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:55.908253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:55.908253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:55.911084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:55.915062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:55.915062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 18 14:53:55.915062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 18 14:53:55.915062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:55.915062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:55.915062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 18 14:53:55.915062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 18 14:53:55.915062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cafe1992d7f34dc1a4791633686a6544. Jul 18 14:53:55.915062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:55.915062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 474/939] 10.222.0.83 () {60 vars in 1140 bytes} [Mon Jul 18 14:53:55 2022] POST /identity/v3/projects => generated 262 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:55.915062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a7192-0c7d-4f12-b437-c04fbca515a2 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:55.921084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:55.979092 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 466/940] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:53:55 2022] PUT /identity/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/692fef632836474da09f0e8a009ffa4e/roles/f7a8b3159cc4475ebac07df3543883cc => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:53:56.030876 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a7192-0c7d-4f12-b437-c04fbca515a2 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:56.080461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5c3aa-1d33-455b-a9ee-5a8f38fbeb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:56.081290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5c3aa-1d33-455b-a9ee-5a8f38fbeb96 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:56.081573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5c3aa-1d33-455b-a9ee-5a8f38fbeb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:56.081849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5c3aa-1d33-455b-a9ee-5a8f38fbeb96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:56.090223 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5c3aa-1d33-455b-a9ee-5a8f38fbeb96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:56.091993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5c3aa-1d33-455b-a9ee-5a8f38fbeb96 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:56.110498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-24c5c3aa-1d33-455b-a9ee-5a8f38fbeb96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:56.110498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:56.110498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:56.110498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:56.110498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:56.110498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:56.110498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:56.110498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:56.110498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:56.110498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:56.110498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:56.110498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:56.110498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:53:56.110498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:53:56.110498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:56.111557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:56.112525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:53:56.112525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:53:56.112525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:53:56.112525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:53:56.112525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:53:56.112525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:53:56.112525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:53:56.112525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:56.117412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 475/941] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:56.118945 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b2282-f89a-4bd4-b76e-f67b85e0fc98 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:56.243840 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b2282-f89a-4bd4-b76e-f67b85e0fc98 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:56.266190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2adc94-909a-4c33-abf6-9d4248351918 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:56.269696 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2adc94-909a-4c33-abf6-9d4248351918 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:56.269696 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2adc94-909a-4c33-abf6-9d4248351918 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:56.271230 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 476/942] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:56.277371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eecc4c-20d3-433b-beb7-ce69fa630471 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:56.277669 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eecc4c-20d3-433b-beb7-ce69fa630471 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:56.277961 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eecc4c-20d3-433b-beb7-ce69fa630471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:56.289582 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e4f67-bf6d-45c1-8d59-750e6d3bac22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:56.290340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e4f67-bf6d-45c1-8d59-750e6d3bac22 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:56.290801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e4f67-bf6d-45c1-8d59-750e6d3bac22 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:56.291187 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e4f67-bf6d-45c1-8d59-750e6d3bac22 None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:56.293018 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e4f67-bf6d-45c1-8d59-750e6d3bac22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:56.293444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e4f67-bf6d-45c1-8d59-750e6d3bac22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:56.295397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e4f67-bf6d-45c1-8d59-750e6d3bac22 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:56.298173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-57eecc4c-20d3-433b-beb7-ce69fa630471 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:56.299787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-57eecc4c-20d3-433b-beb7-ce69fa630471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:56.318387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 467/943] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Jul 18 14:53:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:56.386976 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 477/944] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:53:56.407899 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e4f67-bf6d-45c1-8d59-750e6d3bac22 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:56.421177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:56.529067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e4f67-bf6d-45c1-8d59-750e6d3bac22 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:56.568301 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f797cbd-9ad7-4e3f-8e67-6b1f03c83076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:56.569847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f797cbd-9ad7-4e3f-8e67-6b1f03c83076 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:56.570288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f797cbd-9ad7-4e3f-8e67-6b1f03c83076 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:56.570749 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f797cbd-9ad7-4e3f-8e67-6b1f03c83076 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:56.571666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f797cbd-9ad7-4e3f-8e67-6b1f03c83076 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:56.573269 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f797cbd-9ad7-4e3f-8e67-6b1f03c83076 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-9f797cbd-9ad7-4e3f-8e67-6b1f03c83076 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:56.587695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:56.589060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:56.589060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:56.589060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:56.589060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:56.589060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:56.589060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:56.589060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:53:56.589060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:53:56.589060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:53:56.589060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:53:56.589060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:56.589060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:56.589060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:56.589060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:56.589060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:56.589060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:56.591001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:56.591001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:56.591001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:56.591001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:56.591001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:56.591001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:56.591001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:56.591001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:56.591001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:56.591001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:56.591001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:53:56.591001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:56.591001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:53:56.591001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:56.593751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 468/945] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Jul 18 14:53:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:56.600262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eecc4c-20d3-433b-beb7-ce69fa630471 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:56.609461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:56.711357 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eecc4c-20d3-433b-beb7-ce69fa630471 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:56.749360 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f3f75-0ca0-4cbc-b5f5-ae6808659ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:56.750220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f3f75-0ca0-4cbc-b5f5-ae6808659ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:56.750645 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f3f75-0ca0-4cbc-b5f5-ae6808659ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:56.751040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f3f75-0ca0-4cbc-b5f5-ae6808659ff0 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:56.751898 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f3f75-0ca0-4cbc-b5f5-ae6808659ff0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:56.752319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f3f75-0ca0-4cbc-b5f5-ae6808659ff0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:56.753813 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f3f75-0ca0-4cbc-b5f5-ae6808659ff0 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:56.759881 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 478/946] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Jul 18 14:53:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:56.766062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f797cbd-9ad7-4e3f-8e67-6b1f03c83076 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:56.846192 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:56.882614 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f797cbd-9ad7-4e3f-8e67-6b1f03c83076 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:56.924824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7753e-a893-4ea6-bb5d-75ebf5218b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:56.925507 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7753e-a893-4ea6-bb5d-75ebf5218b27 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:56.925795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7753e-a893-4ea6-bb5d-75ebf5218b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:56.926075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7753e-a893-4ea6-bb5d-75ebf5218b27 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:56.926839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7753e-a893-4ea6-bb5d-75ebf5218b27 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:56.927151 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7753e-a893-4ea6-bb5d-75ebf5218b27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:56.929151 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7753e-a893-4ea6-bb5d-75ebf5218b27 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:56.935274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 469/947] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Jul 18 14:53:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:56.941305 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f3f75-0ca0-4cbc-b5f5-ae6808659ff0 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:57.023237 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:57.058497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f3f75-0ca0-4cbc-b5f5-ae6808659ff0 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:57.099424 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd3ade-64ef-4a39-ac42-93be129ae6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:57.100949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd3ade-64ef-4a39-ac42-93be129ae6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:57.101415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd3ade-64ef-4a39-ac42-93be129ae6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:57.101848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd3ade-64ef-4a39-ac42-93be129ae6a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:57.102990 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd3ade-64ef-4a39-ac42-93be129ae6a4 None admin] RBAC: Adding query filter params (interface=public, service_id=d6fbb13c310f4df6aed7dadee4481ab7, region_id=RegionOne) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:57.103484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd3ade-64ef-4a39-ac42-93be129ae6a4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:57.105030 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd3ade-64ef-4a39-ac42-93be129ae6a4 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:57.111721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 479/948] 10.222.0.83 () {56 vars in 1251 bytes} [Mon Jul 18 14:53:56 2022] GET /identity/v3/endpoints?service_id=d6fbb13c310f4df6aed7dadee4481ab7&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:57.117965 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7753e-a893-4ea6-bb5d-75ebf5218b27 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:57.232793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7753e-a893-4ea6-bb5d-75ebf5218b27 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:57.271086 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5101a-c7c3-4381-93cb-d6ec1cfdd3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:57.272481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5101a-c7c3-4381-93cb-d6ec1cfdd3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:57.272877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5101a-c7c3-4381-93cb-d6ec1cfdd3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:57.273304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5101a-c7c3-4381-93cb-d6ec1cfdd3f7 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:57.274115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5101a-c7c3-4381-93cb-d6ec1cfdd3f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:57.275677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5101a-c7c3-4381-93cb-d6ec1cfdd3f7 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:57.278130 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:57.283996 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 470/949] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Jul 18 14:53:57 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:53:57.344254 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f922a-be78-4064-bdb1-a9b4c66d5cd8 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:57.344832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f922a-be78-4064-bdb1-a9b4c66d5cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:57.344832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f922a-be78-4064-bdb1-a9b4c66d5cd8 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:57.346633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 480/950] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:57.353359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda1454-6a2a-4254-a1f0-8e33e0815d49 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:57.353567 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda1454-6a2a-4254-a1f0-8e33e0815d49 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:57.353960 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda1454-6a2a-4254-a1f0-8e33e0815d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:57.373344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-6dda1454-6a2a-4254-a1f0-8e33e0815d49 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:57.374810 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-6dda1454-6a2a-4254-a1f0-8e33e0815d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:57.464572 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 471/951] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:57.488548 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f922a-be78-4064-bdb1-a9b4c66d5cd8 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:57.650849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f922a-be78-4064-bdb1-a9b4c66d5cd8 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:57.711129 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c500d2-83d5-4db6-9342-e1957b300894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:57.712071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c500d2-83d5-4db6-9342-e1957b300894 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:57.712362 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c500d2-83d5-4db6-9342-e1957b300894 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:57.712670 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c500d2-83d5-4db6-9342-e1957b300894 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:57.713599 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c500d2-83d5-4db6-9342-e1957b300894 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:57.715495 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c500d2-83d5-4db6-9342-e1957b300894 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:57.717958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-d0c500d2-83d5-4db6-9342-e1957b300894 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:57.732980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:57.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:57.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:57.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:57.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:57.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:57.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:57.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:53:57.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:53:57.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:53:57.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:53:57.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:57.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:57.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:57.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:57.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:57.737601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:57.738636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:57.738636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:57.738636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:57.738636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:57.738636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:57.738636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:57.738636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:57.738636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:57.738636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:57.738636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:57.738636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:53:57.738636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:53:57.738636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:53:57.738636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:57.738636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 481/952] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:53:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:57.742589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda1454-6a2a-4254-a1f0-8e33e0815d49 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:57.886093 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda1454-6a2a-4254-a1f0-8e33e0815d49 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:57.928851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:57.936960 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115ae50-35e7-4d73-94d9-34d48709b8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:57.937669 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115ae50-35e7-4d73-94d9-34d48709b8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:57.937970 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115ae50-35e7-4d73-94d9-34d48709b8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:57.938228 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115ae50-35e7-4d73-94d9-34d48709b8a2 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:57.938994 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115ae50-35e7-4d73-94d9-34d48709b8a2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:57.945076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115ae50-35e7-4d73-94d9-34d48709b8a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:57.945076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115ae50-35e7-4d73-94d9-34d48709b8a2 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:57.947444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 472/953] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Jul 18 14:53:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:57.953216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c500d2-83d5-4db6-9342-e1957b300894 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:58.094074 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c500d2-83d5-4db6-9342-e1957b300894 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:58.142716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6262e-148d-4fd8-b69d-31243db1e744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:58.143628 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6262e-148d-4fd8-b69d-31243db1e744 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:58.144095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6262e-148d-4fd8-b69d-31243db1e744 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:58.144500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6262e-148d-4fd8-b69d-31243db1e744 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:58.145276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6262e-148d-4fd8-b69d-31243db1e744 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:58.145587 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6262e-148d-4fd8-b69d-31243db1e744 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:58.147398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6262e-148d-4fd8-b69d-31243db1e744 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:58.153329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 482/954] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Jul 18 14:53:57 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:58.291909 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ded376-9eba-46d8-8497-4cf732bed49c None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:58.292316 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ded376-9eba-46d8-8497-4cf732bed49c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:58.292536 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ded376-9eba-46d8-8497-4cf732bed49c None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:58.294243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 473/955] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:58.300143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27636c6a-f9cc-4adf-a636-dcc3e9f011ed None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:58.300459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27636c6a-f9cc-4adf-a636-dcc3e9f011ed None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:58.300748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27636c6a-f9cc-4adf-a636-dcc3e9f011ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:58.322055 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-27636c6a-f9cc-4adf-a636-dcc3e9f011ed None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:58.323409 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-27636c6a-f9cc-4adf-a636-dcc3e9f011ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:58.392057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:58.403828 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 483/956] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:53:58.433276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ded376-9eba-46d8-8497-4cf732bed49c None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:58.552734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ded376-9eba-46d8-8497-4cf732bed49c None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:58.601309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f660a-ebb9-4c12-aa0d-bdddf958f380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:58.602688 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f660a-ebb9-4c12-aa0d-bdddf958f380 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:58.603335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f660a-ebb9-4c12-aa0d-bdddf958f380 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:58.603887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f660a-ebb9-4c12-aa0d-bdddf958f380 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:58.618250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f660a-ebb9-4c12-aa0d-bdddf958f380 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:53:58.628206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-120f660a-ebb9-4c12-aa0d-bdddf958f380 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:53:58.628206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:58.628206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:58.628206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:58.628206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:58.628206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:58.628206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:58.628206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:58.628206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:58.628206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:58.628206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:58.628206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:58.628206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:53:58.628206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:53:58.628206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:53:58.628206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:53:58.629478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:53:58.629478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:53:58.629478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:53:58.629478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:53:58.629478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:58.629478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:58.629478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:58.629478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:58.629478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:58.629478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:58.629478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:58.629478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:58.629478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:58.629478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:58.629478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:58.629478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:58.630616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:58.630616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:58.630616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:58.630616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:58.630616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:53:58.630616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:53:58.630616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:53:58.630616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:53:58.630616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:53:58.630616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:53:58.630616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:53:58.630616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:58.634226 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 474/957] 10.222.0.83 () {56 vars in 1120 bytes} [Mon Jul 18 14:53:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:58.640260 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27636c6a-f9cc-4adf-a636-dcc3e9f011ed None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:58.758900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27636c6a-f9cc-4adf-a636-dcc3e9f011ed None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:58.782475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edcac5-948f-4faf-9421-dd028c2be47d None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:58.783579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edcac5-948f-4faf-9421-dd028c2be47d None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:58.784254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edcac5-948f-4faf-9421-dd028c2be47d None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:58.786288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 475/958] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:58.792852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9f70b-05a4-404e-8fb0-90674f1cc6c4 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:58.793496 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9f70b-05a4-404e-8fb0-90674f1cc6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:58.794037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9f70b-05a4-404e-8fb0-90674f1cc6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:58.796504 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3abc7f-d963-4ebc-8162-c39bfa564983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:58.797186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3abc7f-d963-4ebc-8162-c39bfa564983 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:58.797476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3abc7f-d963-4ebc-8162-c39bfa564983 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:58.797760 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3abc7f-d963-4ebc-8162-c39bfa564983 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:58.798521 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3abc7f-d963-4ebc-8162-c39bfa564983 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:58.798861 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3abc7f-d963-4ebc-8162-c39bfa564983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:58.800261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3abc7f-d963-4ebc-8162-c39bfa564983 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:58.811115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 484/959] 10.222.0.83 () {56 vars in 1115 bytes} [Mon Jul 18 14:53:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:58.815179 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-4fd9f70b-05a4-404e-8fb0-90674f1cc6c4 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:58.816719 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3abc7f-d963-4ebc-8162-c39bfa564983 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:58.816879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4fd9f70b-05a4-404e-8fb0-90674f1cc6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:58.896912 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:58.911079 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:58.930649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 476/960] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:53:58.950086 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9f70b-05a4-404e-8fb0-90674f1cc6c4 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:58.965041 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3abc7f-d963-4ebc-8162-c39bfa564983 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:59.004492 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1aa6b2-d7c3-4e64-9a9b-261e3bcae757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:59.005879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1aa6b2-d7c3-4e64-9a9b-261e3bcae757 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:59.006949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1aa6b2-d7c3-4e64-9a9b-261e3bcae757 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:59.007709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1aa6b2-d7c3-4e64-9a9b-261e3bcae757 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:59.024031 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1aa6b2-d7c3-4e64-9a9b-261e3bcae757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:59.026272 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1aa6b2-d7c3-4e64-9a9b-261e3bcae757 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:59.053304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-3f1aa6b2-d7c3-4e64-9a9b-261e3bcae757 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:59.053304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:59.053304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:59.053304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:59.053304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:59.053304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:59.053304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:59.053304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:59.053304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:59.053304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:59.053304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:59.053304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:59.053304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:53:59.053304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:53:59.053304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:59.054337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:59.056155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:53:59.056155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:53:59.056155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:53:59.056155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:59.059182 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 485/961] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:59.066085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1aa6b2-d7c3-4e64-9a9b-261e3bcae757 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:59.092993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9f70b-05a4-404e-8fb0-90674f1cc6c4 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:59.141873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa307356-0459-4d2f-9650-91f9bdf12153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:59.142523 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa307356-0459-4d2f-9650-91f9bdf12153 None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:59.142839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa307356-0459-4d2f-9650-91f9bdf12153 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:59.143121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa307356-0459-4d2f-9650-91f9bdf12153 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:59.143846 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa307356-0459-4d2f-9650-91f9bdf12153 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:59.145327 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa307356-0459-4d2f-9650-91f9bdf12153 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:59.147060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: WARNING keystone.server.flask.common [None req-fa307356-0459-4d2f-9650-91f9bdf12153 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 14:53:59.200339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.sql.core [None req-fa307356-0459-4d2f-9650-91f9bdf12153 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 14:53:59.200339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 14:53:59.200339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [parameters: {'user_id': '76e65147a78748d3b0549ed2ed10d208', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 14:53:59.200339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80303) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 14:53:59.210625 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1aa6b2-d7c3-4e64-9a9b-261e3bcae757 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:59.219954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:59.224404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-fa307356-0459-4d2f-9650-91f9bdf12153 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 14:53:59.224404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:59.224404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 14:53:59.224404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 14:53:59.224404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 14:53:59.224404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 14:53:59.224404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 14:53:59.224404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 14:53:59.224404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 14:53:59.224404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application conn.query(q) Jul 18 14:53:59.224404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 14:53:59.224404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 14:53:59.224404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 14:53:59.224404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application result.read() Jul 18 14:53:59.224404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 14:53:59.226474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application next(self.gen) Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application yield resource Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application next(self.gen) Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise self.value Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application session.commit() Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 14:53:59.227526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self.session.flush() Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application compat.raise_( Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 14:53:59.228459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 14:53:59.229481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 14:53:59.229481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 14:53:59.229481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 18 14:53:59.229481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 18 14:53:59.229481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 14:53:59.229481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 14:53:59.229481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 14:53:59.229481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 14:53:59.229481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 14:53:59.229481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 14:53:59.229481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 14:53:59.229481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 14:53:59.229481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 14:53:59.229481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 14:53:59.229481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 14:53:59.229481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception Jul 18 14:53:59.230451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 14:53:59.230451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 14:53:59.230451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 14:53:59.230451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 14:53:59.230451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 14:53:59.230451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 14:53:59.230451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 14:53:59.230451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application conn.query(q) Jul 18 14:53:59.230451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 14:53:59.230451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 14:53:59.230451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 14:53:59.230451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application result.read() Jul 18 14:53:59.230451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 14:53:59.230451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 14:53:59.230451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 14:53:59.230451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 14:53:59.236710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 14:53:59.236710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 14:53:59.236710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 14:53:59.236710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 14:53:59.236710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 14:53:59.236710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 14:53:59.236710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [parameters: {'user_id': '76e65147a78748d3b0549ed2ed10d208', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 14:53:59.236710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 14:53:59.236710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:59.236710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:53:59.236710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:59.236710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:59.236710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:59.236710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:59.236710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 18 14:53:59.238337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 18 14:53:59.239954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 18 14:53:59.239954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 18 14:53:59.239954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 14:53:59.239954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 14:53:59.239954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 14:53:59.239954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:59.239954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 477/962] 10.222.0.83 () {60 vars in 1131 bytes} [Mon Jul 18 14:53:58 2022] POST /identity/v3/users => generated 160 bytes in 283 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:59.239954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa307356-0459-4d2f-9650-91f9bdf12153 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:59.275242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43fbb7-4d25-4d25-9ea4-41e8e447b8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:59.276274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43fbb7-4d25-4d25-9ea4-41e8e447b8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:59.276904 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43fbb7-4d25-4d25-9ea4-41e8e447b8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:59.277475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43fbb7-4d25-4d25-9ea4-41e8e447b8d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:59.278737 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43fbb7-4d25-4d25-9ea4-41e8e447b8d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:59.279244 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43fbb7-4d25-4d25-9ea4-41e8e447b8d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:59.281617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43fbb7-4d25-4d25-9ea4-41e8e447b8d3 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:59.310049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:59.321733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 486/963] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:59.321733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43fbb7-4d25-4d25-9ea4-41e8e447b8d3 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:59.398992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa307356-0459-4d2f-9650-91f9bdf12153 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:59.455557 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9a731-309d-4689-9569-74a95752511a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:59.456561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9a731-309d-4689-9569-74a95752511a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:59.456851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9a731-309d-4689-9569-74a95752511a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:59.457134 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9a731-309d-4689-9569-74a95752511a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:59.467818 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9a731-309d-4689-9569-74a95752511a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:59.469619 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9a731-309d-4689-9569-74a95752511a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:59.474271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43fbb7-4d25-4d25-9ea4-41e8e447b8d3 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:59.487730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-63e9a731-309d-4689-9569-74a95752511a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 14:53:59.487730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:59.487730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:59.487730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:59.487730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:59.487730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:59.487730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:59.487730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:59.487730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:59.487730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:59.487730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:59.487730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:59.487730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:59.487730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:59.487730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:59.487730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:59.488784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:59.489779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:59.489779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:59.489779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:59.489779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:59.489779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:59.489779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:59.489779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:59.489779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:59.489779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 14:53:59.489779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:53:59.489779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 478/964] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:53:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:53:59.495346 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9a731-309d-4689-9569-74a95752511a None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:59.520664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960d3a0-8acd-418f-9505-1709a57c12b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:59.522072 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960d3a0-8acd-418f-9505-1709a57c12b8 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:59.522464 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960d3a0-8acd-418f-9505-1709a57c12b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:59.522779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960d3a0-8acd-418f-9505-1709a57c12b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:59.533303 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960d3a0-8acd-418f-9505-1709a57c12b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:59.535138 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960d3a0-8acd-418f-9505-1709a57c12b8 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:59.556782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-0960d3a0-8acd-418f-9505-1709a57c12b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:53:59.556782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:53:59.556782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:53:59.556782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:53:59.556782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:53:59.556782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:53:59.556782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:53:59.556782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:53:59.556782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:53:59.556782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:53:59.556782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:53:59.556782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:53:59.556782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:53:59.556782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:53:59.556782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:53:59.556782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:53:59.559050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:53:59.560921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:53:59.560921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:53:59.560921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:53:59.560921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:53:59.560921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:53:59.560921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:53:59.560921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:53:59.560921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:53:59.560921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:53:59.560921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:53:59.565307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 487/965] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Jul 18 14:53:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:53:59.565307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960d3a0-8acd-418f-9505-1709a57c12b8 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:59.597058 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:59.645896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9a731-309d-4689-9569-74a95752511a None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:59.655837 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:59.691403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15356cb-e685-42fa-8d6a-1dc66546af74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:59.692545 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15356cb-e685-42fa-8d6a-1dc66546af74 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:59.693047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15356cb-e685-42fa-8d6a-1dc66546af74 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:59.693851 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960d3a0-8acd-418f-9505-1709a57c12b8 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:53:59.694835 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15356cb-e685-42fa-8d6a-1dc66546af74 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:59.697181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15356cb-e685-42fa-8d6a-1dc66546af74 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:59.697864 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15356cb-e685-42fa-8d6a-1dc66546af74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:59.699664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15356cb-e685-42fa-8d6a-1dc66546af74 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:59.736633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbef5e-e036-4840-935c-0f04a9b28b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:53:59.737309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbef5e-e036-4840-935c-0f04a9b28b9c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:59.737592 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbef5e-e036-4840-935c-0f04a9b28b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:59.737868 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbef5e-e036-4840-935c-0f04a9b28b9c None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:59.738779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbef5e-e036-4840-935c-0f04a9b28b9c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:53:59.739071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbef5e-e036-4840-935c-0f04a9b28b9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:53:59.742014 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbef5e-e036-4840-935c-0f04a9b28b9c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:53:59.749172 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 479/966] 10.222.0.83 () {56 vars in 1101 bytes} [Mon Jul 18 14:53:59 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:53:59.749861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45159c-aeb1-4911-a131-d8ea30f7d39e None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:59.750233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45159c-aeb1-4911-a131-d8ea30f7d39e None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:59.750524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45159c-aeb1-4911-a131-d8ea30f7d39e None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:59.752183 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 480/967] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:53:59.758120 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207da4d-9de6-46a1-9edb-5c1cd799ba1a None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:53:59.758477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207da4d-9de6-46a1-9edb-5c1cd799ba1a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:53:59.758730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207da4d-9de6-46a1-9edb-5c1cd799ba1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:53:59.777012 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 488/968] 10.222.0.83 () {56 vars in 1149 bytes} [Mon Jul 18 14:53:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:53:59.778607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-c207da4d-9de6-46a1-9edb-5c1cd799ba1a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:53:59.780275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c207da4d-9de6-46a1-9edb-5c1cd799ba1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:53:59.782858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbef5e-e036-4840-935c-0f04a9b28b9c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:59.872430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 481/969] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:53:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:53:59.896624 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207da4d-9de6-46a1-9edb-5c1cd799ba1a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:53:59.916678 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:53:59.943884 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbef5e-e036-4840-935c-0f04a9b28b9c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:00.008764 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7059cfd-050d-4cf0-89e2-17638d61e5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:00.009445 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7059cfd-050d-4cf0-89e2-17638d61e5ce None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:00.009726 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7059cfd-050d-4cf0-89e2-17638d61e5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:00.010070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7059cfd-050d-4cf0-89e2-17638d61e5ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:00.047268 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:00.056598 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207da4d-9de6-46a1-9edb-5c1cd799ba1a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:00.116552 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7059cfd-050d-4cf0-89e2-17638d61e5ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:00.118052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7059cfd-050d-4cf0-89e2-17638d61e5ce None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:00.121210 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abdbdac-15c0-40d6-aae3-4ed8b3daecf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:00.122528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdbdac-15c0-40d6-aae3-4ed8b3daecf6 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:00.123051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdbdac-15c0-40d6-aae3-4ed8b3daecf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:00.123331 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdbdac-15c0-40d6-aae3-4ed8b3daecf6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:00.124006 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abdbdac-15c0-40d6-aae3-4ed8b3daecf6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:00.125761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abdbdac-15c0-40d6-aae3-4ed8b3daecf6 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-8abdbdac-15c0-40d6-aae3-4ed8b3daecf6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:00.140276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:00.147266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:00.147266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:00.147266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:00.147266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:00.147266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:00.147266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:00.147266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:54:00.147266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:54:00.147266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:54:00.147266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:54:00.147266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:00.147266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:00.147266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:00.147266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:00.147266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:00.147266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:00.148728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:00.148728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:00.148728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:00.148728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:00.148728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:00.148728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:00.148728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:00.148728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:00.148728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:00.148728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:00.148728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:54:00.148728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:54:00.148728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:54:00.148728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:00.148728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 482/970] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Jul 18 14:53:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:54:00.150034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abdbdac-15c0-40d6-aae3-4ed8b3daecf6 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:00.238165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:00.256215 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 489/971] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:53:59 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 14:54:00.262155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7059cfd-050d-4cf0-89e2-17638d61e5ce None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:00.275594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abdbdac-15c0-40d6-aae3-4ed8b3daecf6 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:00.317810 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da25844-3bb9-4f36-a3b2-bb3e1d9cb9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:00.319201 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da25844-3bb9-4f36-a3b2-bb3e1d9cb9ac None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:00.319604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da25844-3bb9-4f36-a3b2-bb3e1d9cb9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:00.319994 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da25844-3bb9-4f36-a3b2-bb3e1d9cb9ac None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:00.321111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da25844-3bb9-4f36-a3b2-bb3e1d9cb9ac None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:00.321944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da25844-3bb9-4f36-a3b2-bb3e1d9cb9ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:00.323573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da25844-3bb9-4f36-a3b2-bb3e1d9cb9ac None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:00.337600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 483/972] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Jul 18 14:54:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:00.338770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da25844-3bb9-4f36-a3b2-bb3e1d9cb9ac None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:00.369058 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:00.384008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7059cfd-050d-4cf0-89e2-17638d61e5ce None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:00.419195 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6dd8a8-0808-408d-967c-666943e1f864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:00.420038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6dd8a8-0808-408d-967c-666943e1f864 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:00.420321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6dd8a8-0808-408d-967c-666943e1f864 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:00.420601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6dd8a8-0808-408d-967c-666943e1f864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:00.432282 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6dd8a8-0808-408d-967c-666943e1f864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:00.434071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6dd8a8-0808-408d-967c-666943e1f864 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:00.452721 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da25844-3bb9-4f36-a3b2-bb3e1d9cb9ac None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:00.454573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-9e6dd8a8-0808-408d-967c-666943e1f864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:54:00.454573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:00.454573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:00.454573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:00.454573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:00.454573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:00.454573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:00.454573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:00.454573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:00.454573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:00.454573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:00.454573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:00.454573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:54:00.454573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:54:00.454573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:00.455838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:00.463144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:54:00.463144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:54:00.463144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:54:00.463144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:00.463144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 490/973] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:54:00.463144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334b9ca-a30d-45b4-b012-7da48c6348f6 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:00.463144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334b9ca-a30d-45b4-b012-7da48c6348f6 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:00.463144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334b9ca-a30d-45b4-b012-7da48c6348f6 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:00.463144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 491/974] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:00.464775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334b9ca-a30d-45b4-b012-7da48c6348f6 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:00.492216 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9193c3-1715-4e40-ab99-108a3c33dc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:00.492885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9193c3-1715-4e40-ab99-108a3c33dc3e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:00.493168 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9193c3-1715-4e40-ab99-108a3c33dc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:00.493446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9193c3-1715-4e40-ab99-108a3c33dc3e None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:00.494177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9193c3-1715-4e40-ab99-108a3c33dc3e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:00.494483 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9193c3-1715-4e40-ab99-108a3c33dc3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:00.495900 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9193c3-1715-4e40-ab99-108a3c33dc3e None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:00.501027 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 484/975] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Jul 18 14:54:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:00.502260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2a45a-fe08-4122-b64f-303729b79cd3 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:00.502558 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2a45a-fe08-4122-b64f-303729b79cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:00.502851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2a45a-fe08-4122-b64f-303729b79cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:00.507212 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:00.520994 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-29e2a45a-fe08-4122-b64f-303729b79cd3 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:00.523190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-29e2a45a-fe08-4122-b64f-303729b79cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:00.573950 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334b9ca-a30d-45b4-b012-7da48c6348f6 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:00.592986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 485/976] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:00.593978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2a45a-fe08-4122-b64f-303729b79cd3 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:00.611890 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae7676-67af-42b8-a62d-a2f78d52041d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:00.612557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae7676-67af-42b8-a62d-a2f78d52041d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:00.612838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae7676-67af-42b8-a62d-a2f78d52041d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:00.613115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae7676-67af-42b8-a62d-a2f78d52041d None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:00.613826 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae7676-67af-42b8-a62d-a2f78d52041d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:00.614132 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae7676-67af-42b8-a62d-a2f78d52041d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:00.615575 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae7676-67af-42b8-a62d-a2f78d52041d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:00.615771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:00.627166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 492/977] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:00.631290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae7676-67af-42b8-a62d-a2f78d52041d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:00.701513 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2a45a-fe08-4122-b64f-303729b79cd3 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:00.726253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:00.732354 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae7676-67af-42b8-a62d-a2f78d52041d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:00.738739 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27df64-364a-4acd-ae92-89c6c5d1ef4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:00.738739 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27df64-364a-4acd-ae92-89c6c5d1ef4f None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:00.738984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27df64-364a-4acd-ae92-89c6c5d1ef4f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:00.739097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27df64-364a-4acd-ae92-89c6c5d1ef4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:00.739843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27df64-364a-4acd-ae92-89c6c5d1ef4f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:00.741329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27df64-364a-4acd-ae92-89c6c5d1ef4f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:00.767890 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e4615-95b3-4295-a57a-e184111e4e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:00.768549 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e4615-95b3-4295-a57a-e184111e4e52 None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:00.768830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e4615-95b3-4295-a57a-e184111e4e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:00.769107 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e4615-95b3-4295-a57a-e184111e4e52 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:00.769817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e4615-95b3-4295-a57a-e184111e4e52 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:00.772046 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e4615-95b3-4295-a57a-e184111e4e52 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:00.830615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 486/978] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Jul 18 14:54:00 2022] POST /identity/v3/endpoints => generated 349 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:54:00.831446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27df64-364a-4acd-ae92-89c6c5d1ef4f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:00.854582 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 493/979] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Jul 18 14:54:00 2022] POST /identity/v3/services => generated 242 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:54:00.941965 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:00.992697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27df64-364a-4acd-ae92-89c6c5d1ef4f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:01.040018 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc44fe1-812c-4986-a96c-b1f0e7af78a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:01.040826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44fe1-812c-4986-a96c-b1f0e7af78a0 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:01.041110 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44fe1-812c-4986-a96c-b1f0e7af78a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:01.041387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44fe1-812c-4986-a96c-b1f0e7af78a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:01.048861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc44fe1-812c-4986-a96c-b1f0e7af78a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:01.050646 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc44fe1-812c-4986-a96c-b1f0e7af78a0 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:01.066740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-3dc44fe1-812c-4986-a96c-b1f0e7af78a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:54:01.066740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:01.066740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:01.066740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:01.066740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:01.066740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:01.066740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:01.066740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:01.066740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:01.066740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:01.066740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:01.066740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:01.066740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:54:01.066740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:54:01.066740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:01.067736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:01.069155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:54:01.069155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:54:01.069155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:54:01.069155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:54:01.069155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:54:01.069155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:54:01.069155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:54:01.069155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:01.069155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 487/980] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:54:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:01.076150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e4615-95b3-4295-a57a-e184111e4e52 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:01.188989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e4615-95b3-4295-a57a-e184111e4e52 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:01.263772 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110379ef-f7c1-4bc0-8773-531d96f56457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:01.263772 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110379ef-f7c1-4bc0-8773-531d96f56457 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:01.263772 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110379ef-f7c1-4bc0-8773-531d96f56457 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:01.263772 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110379ef-f7c1-4bc0-8773-531d96f56457 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:01.264304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110379ef-f7c1-4bc0-8773-531d96f56457 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:01.264304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110379ef-f7c1-4bc0-8773-531d96f56457 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:01.264304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110379ef-f7c1-4bc0-8773-531d96f56457 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:01.264304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 494/981] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:54:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:01.264625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc44fe1-812c-4986-a96c-b1f0e7af78a0 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:01.378989 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc44fe1-812c-4986-a96c-b1f0e7af78a0 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:01.404394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:01.416244 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2c687-204a-4ce1-8874-0f8ecdfc57b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:01.416903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2c687-204a-4ce1-8874-0f8ecdfc57b3 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:01.417213 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2c687-204a-4ce1-8874-0f8ecdfc57b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:01.418406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2c687-204a-4ce1-8874-0f8ecdfc57b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:01.418406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2c687-204a-4ce1-8874-0f8ecdfc57b3 None admin] RBAC: Adding query filter params (role.id=f7a8b3159cc4475ebac07df3543883cc, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=692fef632836474da09f0e8a009ffa4e) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:01.418807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2c687-204a-4ce1-8874-0f8ecdfc57b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:01.420214 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2c687-204a-4ce1-8874-0f8ecdfc57b3 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:01.426661 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 488/982] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:54:01 2022] GET /identity/v3/role_assignments?user.id=692fef632836474da09f0e8a009ffa4e&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=f7a8b3159cc4475ebac07df3543883cc => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:03.000160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ea70c-5a56-4abb-bb1d-001f335b135a None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:03.000724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ea70c-5a56-4abb-bb1d-001f335b135a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:03.000724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ea70c-5a56-4abb-bb1d-001f335b135a None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:03.001075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5465c-bcf6-40cd-afa8-798c3b07a968 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:03.001601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5465c-bcf6-40cd-afa8-798c3b07a968 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:03.001783 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5465c-bcf6-40cd-afa8-798c3b07a968 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:03.002236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 495/983] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:03.003117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 489/984] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:54:03.008742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0b6dc-6732-429f-9d24-10d197056e79 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:03.009032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0b6dc-6732-429f-9d24-10d197056e79 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:03.009316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0b6dc-6732-429f-9d24-10d197056e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:03.009511 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ba5f8-528c-4f35-b93a-a4f4940473ac None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:03.009807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ba5f8-528c-4f35-b93a-a4f4940473ac None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:03.010095 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ba5f8-528c-4f35-b93a-a4f4940473ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:03.027382 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-57c0b6dc-6732-429f-9d24-10d197056e79 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:03.027885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-3e5ba5f8-528c-4f35-b93a-a4f4940473ac None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:03.028597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-57c0b6dc-6732-429f-9d24-10d197056e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:03.028983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3e5ba5f8-528c-4f35-b93a-a4f4940473ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:03.041756 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:03.105285 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 490/985] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:54:03.110910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 496/986] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:03.129245 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0b6dc-6732-429f-9d24-10d197056e79 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:03.138105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ba5f8-528c-4f35-b93a-a4f4940473ac None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:03.244407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0b6dc-6732-429f-9d24-10d197056e79 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:03.253459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ba5f8-528c-4f35-b93a-a4f4940473ac None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:03.275563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:03.283482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baebc8c-d287-4c6e-90e2-921746ac03bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:03.284306 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baebc8c-d287-4c6e-90e2-921746ac03bc None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:03.284618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baebc8c-d287-4c6e-90e2-921746ac03bc None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:03.284896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baebc8c-d287-4c6e-90e2-921746ac03bc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:03.285558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baebc8c-d287-4c6e-90e2-921746ac03bc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:03.287179 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baebc8c-d287-4c6e-90e2-921746ac03bc None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:03.298186 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430ba4a-885d-4978-a455-1be316a7c901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:03.299345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430ba4a-885d-4978-a455-1be316a7c901 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:03.299807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430ba4a-885d-4978-a455-1be316a7c901 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:03.300350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430ba4a-885d-4978-a455-1be316a7c901 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-6baebc8c-d287-4c6e-90e2-921746ac03bc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:03.301073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:03.302095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:03.302095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:03.302095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:03.302095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:03.302095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:03.302095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:03.302095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:54:03.302095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:54:03.302095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:54:03.302095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:54:03.302095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:03.302095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:03.302095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:03.302095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:03.302095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:03.302095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:03.303916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:03.303916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:03.303916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:03.303916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:03.303916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:03.303916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:03.303916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:03.303916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:03.303916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:03.303916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:03.303916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:54:03.303916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:54:03.303916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:54:03.303916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:03.303916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 497/987] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Jul 18 14:54:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:03.305224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430ba4a-885d-4978-a455-1be316a7c901 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:03.306713 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430ba4a-885d-4978-a455-1be316a7c901 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:03.311927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baebc8c-d287-4c6e-90e2-921746ac03bc None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-9430ba4a-885d-4978-a455-1be316a7c901 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:03.320554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:03.321647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:03.321647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:03.321647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:03.321647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:03.321647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:03.321647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:03.321647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:54:03.321647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:54:03.321647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:54:03.321647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:54:03.321647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:03.321647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:03.321647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:03.321647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:03.321647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:03.321647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:03.322722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:03.322722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:03.322722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:03.322722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:03.322722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:03.322722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:03.322722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:03.322722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:03.322722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:03.322722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:03.322722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:54:03.322722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:54:03.322722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:54:03.322722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:03.326809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 491/988] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:54:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:54:03.328912 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430ba4a-885d-4978-a455-1be316a7c901 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:03.329476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:03.439145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baebc8c-d287-4c6e-90e2-921746ac03bc None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:03.442584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430ba4a-885d-4978-a455-1be316a7c901 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:03.476834 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94821a63-6352-4e8b-b894-cca7a9b1632d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:03.477658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94821a63-6352-4e8b-b894-cca7a9b1632d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:03.478013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94821a63-6352-4e8b-b894-cca7a9b1632d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:03.478364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94821a63-6352-4e8b-b894-cca7a9b1632d None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:03.479213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94821a63-6352-4e8b-b894-cca7a9b1632d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:03.479601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94821a63-6352-4e8b-b894-cca7a9b1632d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:03.480237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee55946-fa96-483b-ba77-683e346c3630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:03.481155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee55946-fa96-483b-ba77-683e346c3630 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:03.481436 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94821a63-6352-4e8b-b894-cca7a9b1632d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:03.481552 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee55946-fa96-483b-ba77-683e346c3630 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:03.481709 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee55946-fa96-483b-ba77-683e346c3630 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:03.482419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee55946-fa96-483b-ba77-683e346c3630 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:03.482762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee55946-fa96-483b-ba77-683e346c3630 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:03.484118 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee55946-fa96-483b-ba77-683e346c3630 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:03.487374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 498/989] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Jul 18 14:54:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:03.494813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 492/990] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Jul 18 14:54:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:03.494978 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94821a63-6352-4e8b-b894-cca7a9b1632d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:03.495834 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee55946-fa96-483b-ba77-683e346c3630 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:03.547651 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:03.575440 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:03.599355 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94821a63-6352-4e8b-b894-cca7a9b1632d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:03.604130 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee55946-fa96-483b-ba77-683e346c3630 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:03.634887 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228de6d-4c43-4d48-86ca-6f9a38be3e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:03.635569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228de6d-4c43-4d48-86ca-6f9a38be3e35 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:03.635860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228de6d-4c43-4d48-86ca-6f9a38be3e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:03.636142 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228de6d-4c43-4d48-86ca-6f9a38be3e35 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:03.636862 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228de6d-4c43-4d48-86ca-6f9a38be3e35 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:03.637209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228de6d-4c43-4d48-86ca-6f9a38be3e35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:03.638630 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228de6d-4c43-4d48-86ca-6f9a38be3e35 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:03.640734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e943b-1682-4b20-9a36-a583566b0791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:03.641405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e943b-1682-4b20-9a36-a583566b0791 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:03.641693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e943b-1682-4b20-9a36-a583566b0791 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:03.641973 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e943b-1682-4b20-9a36-a583566b0791 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:03.642752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e943b-1682-4b20-9a36-a583566b0791 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:03.643060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e943b-1682-4b20-9a36-a583566b0791 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:03.643733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 499/991] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Jul 18 14:54:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:03.644439 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e943b-1682-4b20-9a36-a583566b0791 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:03.644992 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722915fc-17eb-498d-ac0b-569e014d5330 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:03.645283 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722915fc-17eb-498d-ac0b-569e014d5330 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:03.645543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722915fc-17eb-498d-ac0b-569e014d5330 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:03.647136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 500/992] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:03.650723 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 493/993] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Jul 18 14:54:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:03.651918 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722915fc-17eb-498d-ac0b-569e014d5330 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:03.652766 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7548de-0aa8-452c-9bc8-4e355c3562ca None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:03.653157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7548de-0aa8-452c-9bc8-4e355c3562ca None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:03.653507 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7548de-0aa8-452c-9bc8-4e355c3562ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:03.670945 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-5d7548de-0aa8-452c-9bc8-4e355c3562ca None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:03.672263 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-5d7548de-0aa8-452c-9bc8-4e355c3562ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:03.742510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 494/994] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:03.743691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7548de-0aa8-452c-9bc8-4e355c3562ca None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:03.763594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722915fc-17eb-498d-ac0b-569e014d5330 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:03.790955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:03.800879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76523df-7492-469f-99e5-7e075b057251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:03.801613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76523df-7492-469f-99e5-7e075b057251 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:03.802640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76523df-7492-469f-99e5-7e075b057251 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:03.802640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76523df-7492-469f-99e5-7e075b057251 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:03.803208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76523df-7492-469f-99e5-7e075b057251 None admin] RBAC: Adding query filter params (interface=admin, service_id=d6fbb13c310f4df6aed7dadee4481ab7, region_id=RegionOne) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:03.803525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76523df-7492-469f-99e5-7e075b057251 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:03.804921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76523df-7492-469f-99e5-7e075b057251 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:03.810305 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 501/995] 10.222.0.83 () {56 vars in 1249 bytes} [Mon Jul 18 14:54:03 2022] GET /identity/v3/endpoints?service_id=d6fbb13c310f4df6aed7dadee4481ab7&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:03.810952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76523df-7492-469f-99e5-7e075b057251 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:03.817614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:03.851654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7548de-0aa8-452c-9bc8-4e355c3562ca None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:03.891380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c933ec-813e-4c8c-bd4f-946170143a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:03.892049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c933ec-813e-4c8c-bd4f-946170143a67 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:03.892338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c933ec-813e-4c8c-bd4f-946170143a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:03.892620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c933ec-813e-4c8c-bd4f-946170143a67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:03.893419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c933ec-813e-4c8c-bd4f-946170143a67 None admin] RBAC: Adding query filter params (interface=public, service_id=b509a32958714d19ad8c6f01dd10d6f2, region_id=RegionOne) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:03.893739 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c933ec-813e-4c8c-bd4f-946170143a67 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:03.895185 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c933ec-813e-4c8c-bd4f-946170143a67 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:03.901507 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 495/996] 10.222.0.83 () {56 vars in 1251 bytes} [Mon Jul 18 14:54:03 2022] GET /identity/v3/endpoints?service_id=b509a32958714d19ad8c6f01dd10d6f2&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:03.902304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c933ec-813e-4c8c-bd4f-946170143a67 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:03.921510 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76523df-7492-469f-99e5-7e075b057251 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:03.961879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641da844-8a12-4ac8-bf14-6659be6fbabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:03.962714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641da844-8a12-4ac8-bf14-6659be6fbabd None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:03.963016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641da844-8a12-4ac8-bf14-6659be6fbabd None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:03.963278 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641da844-8a12-4ac8-bf14-6659be6fbabd None admin] PATH_INFO: `/v3/services/image` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:03.963940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641da844-8a12-4ac8-bf14-6659be6fbabd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:03.965329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641da844-8a12-4ac8-bf14-6659be6fbabd None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-641da844-8a12-4ac8-bf14-6659be6fbabd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:03.977216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:03.978395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:03.978395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:03.978395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:03.978395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:03.978395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:03.978395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:03.978395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:54:03.978395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:54:03.978395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:54:03.978395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:54:03.978395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:03.978395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:03.978395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:03.978395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:03.978395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:03.978395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:03.979745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:03.979745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:03.979745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:03.979745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:03.979745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:03.979745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:03.979745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:03.979745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:03.979745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:03.979745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:03.979745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:54:03.979745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:54:03.979745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:54:03.979745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:03.979745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 502/997] 10.222.0.83 () {56 vars in 1105 bytes} [Mon Jul 18 14:54:03 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:03.980991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641da844-8a12-4ac8-bf14-6659be6fbabd None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:04.015606 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c933ec-813e-4c8c-bd4f-946170143a67 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:04.017815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:04.049518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:04.050786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a89e73-7b29-4560-b4f0-9ca008f5f98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:04.051471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a89e73-7b29-4560-b4f0-9ca008f5f98e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:04.052020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a89e73-7b29-4560-b4f0-9ca008f5f98e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:04.052841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a89e73-7b29-4560-b4f0-9ca008f5f98e None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:04.053933 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a89e73-7b29-4560-b4f0-9ca008f5f98e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:04.055734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a89e73-7b29-4560-b4f0-9ca008f5f98e None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:04.062237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 496/998] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Jul 18 14:54:03 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:54:04.063094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a89e73-7b29-4560-b4f0-9ca008f5f98e None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:04.092371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641da844-8a12-4ac8-bf14-6659be6fbabd None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:04.141088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3445249-a101-4684-97c3-cd60b4c91ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:04.141776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3445249-a101-4684-97c3-cd60b4c91ccb None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:04.142061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3445249-a101-4684-97c3-cd60b4c91ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:04.142341 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3445249-a101-4684-97c3-cd60b4c91ccb None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:04.143087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3445249-a101-4684-97c3-cd60b4c91ccb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:04.144506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3445249-a101-4684-97c3-cd60b4c91ccb None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:04.166574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 503/999] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Jul 18 14:54:03 2022] GET /identity/v3/services => generated 1727 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:54:04.201658 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a89e73-7b29-4560-b4f0-9ca008f5f98e None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:04.248467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede96849-3471-4644-9792-6986a19b2fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:04.249177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede96849-3471-4644-9792-6986a19b2fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:04.249470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede96849-3471-4644-9792-6986a19b2fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:04.249750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede96849-3471-4644-9792-6986a19b2fa0 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:04.250507 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede96849-3471-4644-9792-6986a19b2fa0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:04.250880 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede96849-3471-4644-9792-6986a19b2fa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:04.252352 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede96849-3471-4644-9792-6986a19b2fa0 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:04.258697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 497/1000] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:04.265029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3445249-a101-4684-97c3-cd60b4c91ccb None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:04.388363 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3445249-a101-4684-97c3-cd60b4c91ccb None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:04.419898 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:04.425898 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617e64d-baa6-4ca8-afaa-cde21f990070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:04.426597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617e64d-baa6-4ca8-afaa-cde21f990070 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:04.426885 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617e64d-baa6-4ca8-afaa-cde21f990070 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:04.427234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617e64d-baa6-4ca8-afaa-cde21f990070 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:04.428801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617e64d-baa6-4ca8-afaa-cde21f990070 None admin] RBAC: Adding query filter params (type=image) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:04.429231 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617e64d-baa6-4ca8-afaa-cde21f990070 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:04.430639 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617e64d-baa6-4ca8-afaa-cde21f990070 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:04.436317 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 504/1001] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:04 2022] GET /identity/v3/services?type=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:06.173062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dac4b8-b307-4290-848c-a8caca4b5a9c None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:06.173668 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dac4b8-b307-4290-848c-a8caca4b5a9c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:06.173668 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dac4b8-b307-4290-848c-a8caca4b5a9c None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:06.175230 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 498/1002] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:06.180939 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8babab-d69c-4ae0-b11e-88e247bfe90c None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:06.181470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8babab-d69c-4ae0-b11e-88e247bfe90c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:06.181470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8babab-d69c-4ae0-b11e-88e247bfe90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:06.200141 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-2c8babab-d69c-4ae0-b11e-88e247bfe90c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:06.201360 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2c8babab-d69c-4ae0-b11e-88e247bfe90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:06.280491 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b3768-939c-4894-88cb-415e149e1adb None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:06.294916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b3768-939c-4894-88cb-415e149e1adb None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:06.294916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b3768-939c-4894-88cb-415e149e1adb None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:06.294916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 499/1003] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:06.295209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 505/1004] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:06.295308 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e6800-69f4-47d7-8ff8-00a323a8af2c None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:06.295308 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e6800-69f4-47d7-8ff8-00a323a8af2c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:06.295308 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e6800-69f4-47d7-8ff8-00a323a8af2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:06.312621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-fe1e6800-69f4-47d7-8ff8-00a323a8af2c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:06.313904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-fe1e6800-69f4-47d7-8ff8-00a323a8af2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:06.320690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8babab-d69c-4ae0-b11e-88e247bfe90c None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:06.346003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:06.398595 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 500/1005] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:06.420139 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e6800-69f4-47d7-8ff8-00a323a8af2c None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:06.434030 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8babab-d69c-4ae0-b11e-88e247bfe90c None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:06.462718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:06.477153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01141c-3a02-4aed-8569-bd0f951f3754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:06.478248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01141c-3a02-4aed-8569-bd0f951f3754 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:06.478248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01141c-3a02-4aed-8569-bd0f951f3754 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:06.478561 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01141c-3a02-4aed-8569-bd0f951f3754 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:06.479637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01141c-3a02-4aed-8569-bd0f951f3754 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:06.481803 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01141c-3a02-4aed-8569-bd0f951f3754 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-0b01141c-3a02-4aed-8569-bd0f951f3754 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:06.498122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:06.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:06.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:06.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:06.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:06.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:06.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:06.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:54:06.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:54:06.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:54:06.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:54:06.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:06.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:06.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:06.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:06.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:06.499384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:06.500476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:06.500476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:06.500476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:06.500476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:06.500476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:06.500476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:06.500476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:06.500476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:06.500476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:06.500476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:06.500476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:54:06.500476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:54:06.500476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:54:06.500476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:06.502872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 506/1006] 10.222.0.83 () {56 vars in 1126 bytes} [Mon Jul 18 14:54:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:06.505283 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01141c-3a02-4aed-8569-bd0f951f3754 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:06.531791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e6800-69f4-47d7-8ff8-00a323a8af2c None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:06.568772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f8361-6682-4a94-840f-5ddeb00b7121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:06.569585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f8361-6682-4a94-840f-5ddeb00b7121 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:06.569876 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f8361-6682-4a94-840f-5ddeb00b7121 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:06.570152 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f8361-6682-4a94-840f-5ddeb00b7121 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:06.570847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f8361-6682-4a94-840f-5ddeb00b7121 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:06.572238 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f8361-6682-4a94-840f-5ddeb00b7121 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-f91f8361-6682-4a94-840f-5ddeb00b7121 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:06.584353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:06.585522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:06.585522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:06.585522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:06.585522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:06.585522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:06.585522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:06.585522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:54:06.585522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:54:06.585522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:54:06.585522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:54:06.585522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:06.585522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:06.585522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:06.585522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:06.585522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:06.585522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:06.586387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:06.586387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:06.586387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:06.586387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:06.586387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:06.586387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:06.586387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:06.586387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:06.586387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:06.586387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:06.586387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:54:06.586387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:54:06.586387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:54:06.586387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:06.586387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 501/1007] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:54:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:54:06.587360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2fd5d-f0f9-4e04-a2de-ca153c893eba None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:06.588369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2fd5d-f0f9-4e04-a2de-ca153c893eba None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:06.588369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2fd5d-f0f9-4e04-a2de-ca153c893eba None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:06.589330 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 502/1008] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:06.593201 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2fd5d-f0f9-4e04-a2de-ca153c893eba None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:06.615384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01141c-3a02-4aed-8569-bd0f951f3754 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:06.622999 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:06.650496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d626a-da7c-4725-919c-532b0031c1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:06.651193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d626a-da7c-4725-919c-532b0031c1df None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:06.651481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d626a-da7c-4725-919c-532b0031c1df None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:06.651761 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d626a-da7c-4725-919c-532b0031c1df None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:06.652470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d626a-da7c-4725-919c-532b0031c1df None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:06.652780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d626a-da7c-4725-919c-532b0031c1df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:06.654664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d626a-da7c-4725-919c-532b0031c1df None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:06.660202 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 507/1009] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Jul 18 14:54:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:06.661293 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0dde0-0a0f-4d54-bb84-3bc95e3cc801 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:06.662132 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0dde0-0a0f-4d54-bb84-3bc95e3cc801 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:06.662417 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0dde0-0a0f-4d54-bb84-3bc95e3cc801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:06.679980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-7bb0dde0-0a0f-4d54-bb84-3bc95e3cc801 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:06.681181 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7bb0dde0-0a0f-4d54-bb84-3bc95e3cc801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:06.705655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2fd5d-f0f9-4e04-a2de-ca153c893eba None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:06.729392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:06.743298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87860f-cd4e-423e-8116-5f71d67e2a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:06.743970 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87860f-cd4e-423e-8116-5f71d67e2a62 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:06.744254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87860f-cd4e-423e-8116-5f71d67e2a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:06.744529 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87860f-cd4e-423e-8116-5f71d67e2a62 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:06.745240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87860f-cd4e-423e-8116-5f71d67e2a62 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:06.745543 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87860f-cd4e-423e-8116-5f71d67e2a62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:06.746979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87860f-cd4e-423e-8116-5f71d67e2a62 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:06.750972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 508/1010] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:06.751895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0dde0-0a0f-4d54-bb84-3bc95e3cc801 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:06.752635 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 503/1011] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Jul 18 14:54:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:06.758326 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87860f-cd4e-423e-8116-5f71d67e2a62 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:06.853341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:06.859359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0dde0-0a0f-4d54-bb84-3bc95e3cc801 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:06.863458 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87860f-cd4e-423e-8116-5f71d67e2a62 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:06.895268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231bf5b-5a70-441b-9ff5-f5169b3c8ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:06.896155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231bf5b-5a70-441b-9ff5-f5169b3c8ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:06.896817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231bf5b-5a70-441b-9ff5-f5169b3c8ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:06.896817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231bf5b-5a70-441b-9ff5-f5169b3c8ff6 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:06.897352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231bf5b-5a70-441b-9ff5-f5169b3c8ff6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:06.897659 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231bf5b-5a70-441b-9ff5-f5169b3c8ff6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:06.899063 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231bf5b-5a70-441b-9ff5-f5169b3c8ff6 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:06.899978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87f171-64f9-4b7a-af58-1b6241ca15bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:06.900653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87f171-64f9-4b7a-af58-1b6241ca15bf None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:06.900942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87f171-64f9-4b7a-af58-1b6241ca15bf None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:06.901222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87f171-64f9-4b7a-af58-1b6241ca15bf None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:06.901929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87f171-64f9-4b7a-af58-1b6241ca15bf None admin] RBAC: Adding query filter params (type=compute) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:06.902238 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87f171-64f9-4b7a-af58-1b6241ca15bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:06.903660 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87f171-64f9-4b7a-af58-1b6241ca15bf None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:06.904231 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 509/1012] 10.222.0.83 () {56 vars in 1122 bytes} [Mon Jul 18 14:54:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:06.904918 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231bf5b-5a70-441b-9ff5-f5169b3c8ff6 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:06.909241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 504/1013] 10.222.0.83 () {56 vars in 1112 bytes} [Mon Jul 18 14:54:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:06.910025 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87f171-64f9-4b7a-af58-1b6241ca15bf None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:06.957273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:07.013122 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87f171-64f9-4b7a-af58-1b6241ca15bf None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:07.017145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231bf5b-5a70-441b-9ff5-f5169b3c8ff6 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:07.050046 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f749a79-412c-4994-b626-8daaaaa8acdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:07.050755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f749a79-412c-4994-b626-8daaaaa8acdd None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:07.051043 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f749a79-412c-4994-b626-8daaaaa8acdd None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:07.051322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f749a79-412c-4994-b626-8daaaaa8acdd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:07.052608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f749a79-412c-4994-b626-8daaaaa8acdd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:07.054013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f749a79-412c-4994-b626-8daaaaa8acdd None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:07.055107 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99740a-a42d-46a4-82a7-0952fe80300d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:07.055785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99740a-a42d-46a4-82a7-0952fe80300d None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:07.056073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99740a-a42d-46a4-82a7-0952fe80300d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:07.056359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99740a-a42d-46a4-82a7-0952fe80300d None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:07.057118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99740a-a42d-46a4-82a7-0952fe80300d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:07.058503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99740a-a42d-46a4-82a7-0952fe80300d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:07.128185 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 505/1014] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Jul 18 14:54:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:54:07.129016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f749a79-412c-4994-b626-8daaaaa8acdd None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:07.149017 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:07.173599 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 510/1015] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Jul 18 14:54:06 2022] POST /identity/v3/services => generated 242 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:54:07.273494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f749a79-412c-4994-b626-8daaaaa8acdd None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:07.361018 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602cd3c-d371-43dc-b1b1-0e46ecc2c423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:07.361716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602cd3c-d371-43dc-b1b1-0e46ecc2c423 None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:07.362016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602cd3c-d371-43dc-b1b1-0e46ecc2c423 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:07.362304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602cd3c-d371-43dc-b1b1-0e46ecc2c423 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:07.363193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602cd3c-d371-43dc-b1b1-0e46ecc2c423 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:07.364701 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602cd3c-d371-43dc-b1b1-0e46ecc2c423 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:07.438072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 506/1016] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Jul 18 14:54:07 2022] POST /identity/v3/endpoints => generated 334 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:54:09.310218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4326e1a-32d9-4d90-aa52-d6c79a6e0eb9 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:09.310810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4326e1a-32d9-4d90-aa52-d6c79a6e0eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:09.310810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4326e1a-32d9-4d90-aa52-d6c79a6e0eb9 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:09.311249 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797aa80-3933-451b-ab45-724e59b34813 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:09.311636 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797aa80-3933-451b-ab45-724e59b34813 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:09.311714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797aa80-3933-451b-ab45-724e59b34813 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:09.312338 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 511/1017] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:09.313324 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 507/1018] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:09.318367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9827f4-ade2-4a26-9d14-440eb7715a20 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:09.318367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9827f4-ade2-4a26-9d14-440eb7715a20 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:09.318721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9827f4-ade2-4a26-9d14-440eb7715a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:09.319944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b35ba-c07d-4640-b289-32a14269137e None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:09.320263 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b35ba-c07d-4640-b289-32a14269137e None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:09.320593 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b35ba-c07d-4640-b289-32a14269137e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:09.336745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-5d9827f4-ade2-4a26-9d14-440eb7715a20 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:09.338016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-5d9827f4-ade2-4a26-9d14-440eb7715a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:09.340281 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-706b35ba-c07d-4640-b289-32a14269137e None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:09.341470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-706b35ba-c07d-4640-b289-32a14269137e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:09.405544 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:09.419787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 512/1019] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:09.428675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 508/1020] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:54:09.444269 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9827f4-ade2-4a26-9d14-440eb7715a20 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:09.452945 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b35ba-c07d-4640-b289-32a14269137e None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:09.478833 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:09.555312 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9827f4-ade2-4a26-9d14-440eb7715a20 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:09.577209 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b35ba-c07d-4640-b289-32a14269137e None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:09.593018 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3852592-17ff-4f69-a47a-f556a079a032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:09.593820 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3852592-17ff-4f69-a47a-f556a079a032 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:09.594105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3852592-17ff-4f69-a47a-f556a079a032 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:09.594377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3852592-17ff-4f69-a47a-f556a079a032 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:09.595660 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3852592-17ff-4f69-a47a-f556a079a032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:09.597204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3852592-17ff-4f69-a47a-f556a079a032 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:09.599596 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 513/1021] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:09.605526 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3852592-17ff-4f69-a47a-f556a079a032 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:09.620511 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f56fe-4a83-4d46-abf4-66b986a1e0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:09.621319 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f56fe-4a83-4d46-abf4-66b986a1e0de None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:09.621601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f56fe-4a83-4d46-abf4-66b986a1e0de None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:09.621875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f56fe-4a83-4d46-abf4-66b986a1e0de None admin] PATH_INFO: `/v3/services/image` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:09.622580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f56fe-4a83-4d46-abf4-66b986a1e0de None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:09.623986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f56fe-4a83-4d46-abf4-66b986a1e0de None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-e14f56fe-4a83-4d46-abf4-66b986a1e0de None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:09.636648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:09.637754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:09.637754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:09.637754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:09.637754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:09.637754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:09.637754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:09.637754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:54:09.637754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:54:09.637754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:54:09.637754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:54:09.637754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:09.637754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:09.637754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:09.637754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:09.637754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:09.637754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:09.639076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:09.639076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:09.639076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:09.639076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:09.639076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:09.639076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:09.639076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:09.639076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:09.639076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:09.639076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:09.639076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:54:09.639076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:54:09.639076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:54:09.639076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:09.639076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 509/1022] 10.222.0.83 () {56 vars in 1105 bytes} [Mon Jul 18 14:54:09 2022] GET /identity/v3/services/image => generated 86 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:54:09.643813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f56fe-4a83-4d46-abf4-66b986a1e0de None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:09.705050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:09.714152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:09.717354 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3852592-17ff-4f69-a47a-f556a079a032 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:09.752716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35990cb2-c18a-48cd-9600-b66d840e1872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:09.753382 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35990cb2-c18a-48cd-9600-b66d840e1872 None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:09.753682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35990cb2-c18a-48cd-9600-b66d840e1872 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:09.753961 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35990cb2-c18a-48cd-9600-b66d840e1872 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:09.754710 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35990cb2-c18a-48cd-9600-b66d840e1872 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:09.754958 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f56fe-4a83-4d46-abf4-66b986a1e0de None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:09.756220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35990cb2-c18a-48cd-9600-b66d840e1872 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:09.791177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc2b9d-dc1b-44b8-b65e-6ae3bbd1386a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:09.791944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc2b9d-dc1b-44b8-b65e-6ae3bbd1386a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:09.792261 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc2b9d-dc1b-44b8-b65e-6ae3bbd1386a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:09.792539 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc2b9d-dc1b-44b8-b65e-6ae3bbd1386a None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:09.793255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc2b9d-dc1b-44b8-b65e-6ae3bbd1386a None admin] RBAC: Adding query filter params (name=image) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:09.793564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc2b9d-dc1b-44b8-b65e-6ae3bbd1386a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:09.794989 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc2b9d-dc1b-44b8-b65e-6ae3bbd1386a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:09.800080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 510/1023] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:09.801291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d88e55-2643-4726-acd2-b961ee3fba6b None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:09.801579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d88e55-2643-4726-acd2-b961ee3fba6b None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:09.801945 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d88e55-2643-4726-acd2-b961ee3fba6b None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:09.803220 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 511/1024] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:09.806940 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d88e55-2643-4726-acd2-b961ee3fba6b None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:09.886682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 514/1025] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Jul 18 14:54:09 2022] POST /identity/v3/projects => generated 311 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:54:09.886682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fad3cc-7ef2-4d6f-bfbb-ee70ea0d4690 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:09.886682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fad3cc-7ef2-4d6f-bfbb-ee70ea0d4690 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:09.886682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fad3cc-7ef2-4d6f-bfbb-ee70ea0d4690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:09.896125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-a6fad3cc-7ef2-4d6f-bfbb-ee70ea0d4690 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:09.901262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a6fad3cc-7ef2-4d6f-bfbb-ee70ea0d4690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:09.919706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d88e55-2643-4726-acd2-b961ee3fba6b None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:09.943357 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:09.964701 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea949ab-223d-4fdf-b3b9-c3b5a2004ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:09.965396 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea949ab-223d-4fdf-b3b9-c3b5a2004ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:09.965680 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea949ab-223d-4fdf-b3b9-c3b5a2004ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:09.965954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea949ab-223d-4fdf-b3b9-c3b5a2004ba2 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:09.966795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea949ab-223d-4fdf-b3b9-c3b5a2004ba2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:09.967102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea949ab-223d-4fdf-b3b9-c3b5a2004ba2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:09.968534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea949ab-223d-4fdf-b3b9-c3b5a2004ba2 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:09.980848 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 512/1026] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:09.985161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea949ab-223d-4fdf-b3b9-c3b5a2004ba2 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:09.993810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 515/1027] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:10.021284 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fad3cc-7ef2-4d6f-bfbb-ee70ea0d4690 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:10.135827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea949ab-223d-4fdf-b3b9-c3b5a2004ba2 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:10.147956 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:10.169390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fad3cc-7ef2-4d6f-bfbb-ee70ea0d4690 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:10.178562 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191da78f-6e54-490c-afc3-6292f1d18a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:10.179344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191da78f-6e54-490c-afc3-6292f1d18a78 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:10.179700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191da78f-6e54-490c-afc3-6292f1d18a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:10.179986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191da78f-6e54-490c-afc3-6292f1d18a78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:10.180802 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191da78f-6e54-490c-afc3-6292f1d18a78 None admin] RBAC: Adding query filter params (interface=public, service_id=2711ec7d77b94ba79cbda29a180f3705, region_id=RegionOne) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:10.181179 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191da78f-6e54-490c-afc3-6292f1d18a78 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:10.182564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191da78f-6e54-490c-afc3-6292f1d18a78 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:10.188344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 513/1028] 10.222.0.83 () {56 vars in 1251 bytes} [Mon Jul 18 14:54:09 2022] GET /identity/v3/endpoints?service_id=2711ec7d77b94ba79cbda29a180f3705&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:10.194603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191da78f-6e54-490c-afc3-6292f1d18a78 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:10.216462 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af7330-e461-42dd-8d77-61524ae83f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:10.217585 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af7330-e461-42dd-8d77-61524ae83f34 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:10.217922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af7330-e461-42dd-8d77-61524ae83f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:10.218285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af7330-e461-42dd-8d77-61524ae83f34 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:10.231268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af7330-e461-42dd-8d77-61524ae83f34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:54:10.239637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-83af7330-e461-42dd-8d77-61524ae83f34 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:54:10.239637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:10.239637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:10.239637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:10.239637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:10.239637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:10.239637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:10.239637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:10.239637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:10.239637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:10.239637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:10.239637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:10.239637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:54:10.239637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:54:10.239637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:54:10.239637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:54:10.240779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:54:10.240779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:54:10.240779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:54:10.240779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:54:10.240779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:10.240779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:10.240779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:10.240779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:10.240779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:10.240779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:10.240779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:10.240779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:10.240779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:10.240779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:10.240779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:10.240779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:10.242432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:10.242432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:10.242432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:10.242432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:10.242432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:54:10.242432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:54:10.242432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:54:10.242432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:54:10.242432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:54:10.242432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:54:10.242432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:54:10.242432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:10.242432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 516/1029] 10.222.0.83 () {56 vars in 1120 bytes} [Mon Jul 18 14:54:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:54:10.247179 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af7330-e461-42dd-8d77-61524ae83f34 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:10.250863 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:10.323436 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191da78f-6e54-490c-afc3-6292f1d18a78 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:10.368512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f418994-5cd2-4219-846c-d431710230b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:10.369806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f418994-5cd2-4219-846c-d431710230b7 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:10.370177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f418994-5cd2-4219-846c-d431710230b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:10.370514 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f418994-5cd2-4219-846c-d431710230b7 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:10.371321 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f418994-5cd2-4219-846c-d431710230b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:10.372736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f418994-5cd2-4219-846c-d431710230b7 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:10.372917 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af7330-e461-42dd-8d77-61524ae83f34 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:10.381083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 514/1030] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Jul 18 14:54:10 2022] GET /identity/v3/services => generated 1957 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:54:10.416248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedc6ed-76fd-496b-a775-a0af6a28c077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:10.416248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedc6ed-76fd-496b-a775-a0af6a28c077 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:10.416248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedc6ed-76fd-496b-a775-a0af6a28c077 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:10.416248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedc6ed-76fd-496b-a775-a0af6a28c077 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:10.416865 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedc6ed-76fd-496b-a775-a0af6a28c077 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:10.417040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedc6ed-76fd-496b-a775-a0af6a28c077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:10.418742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedc6ed-76fd-496b-a775-a0af6a28c077 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:10.428537 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 517/1031] 10.222.0.83 () {56 vars in 1115 bytes} [Mon Jul 18 14:54:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:10.434780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f418994-5cd2-4219-846c-d431710230b7 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:10.563489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f418994-5cd2-4219-846c-d431710230b7 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:10.575031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:10.602755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbbc39-21b3-4209-a02b-0fe12693b267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:10.603571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbbc39-21b3-4209-a02b-0fe12693b267 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:10.603850 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbbc39-21b3-4209-a02b-0fe12693b267 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:10.604126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbbc39-21b3-4209-a02b-0fe12693b267 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:10.615471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbbc39-21b3-4209-a02b-0fe12693b267 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:10.616947 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbbc39-21b3-4209-a02b-0fe12693b267 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:10.637091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-6fbbbc39-21b3-4209-a02b-0fe12693b267 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:54:10.637091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:10.637091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:10.637091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:10.637091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:10.637091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:10.637091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:10.637091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:10.637091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:10.637091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:10.637091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:10.637091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:10.637091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:54:10.637091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:54:10.637091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:10.638526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:10.639590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:54:10.639590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:54:10.639590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:54:10.639590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:10.642630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 515/1032] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:10.644611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedc6ed-76fd-496b-a775-a0af6a28c077 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:10.649194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:10.749307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedc6ed-76fd-496b-a775-a0af6a28c077 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:10.785278 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5513939-63a4-4795-be2a-4aafc7912ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:10.785972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5513939-63a4-4795-be2a-4aafc7912ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:10.786995 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5513939-63a4-4795-be2a-4aafc7912ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:10.787131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5513939-63a4-4795-be2a-4aafc7912ae4 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:10.787848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5513939-63a4-4795-be2a-4aafc7912ae4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:10.788159 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5513939-63a4-4795-be2a-4aafc7912ae4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:10.789597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5513939-63a4-4795-be2a-4aafc7912ae4 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:10.805391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 518/1033] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:10.811293 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbbc39-21b3-4209-a02b-0fe12693b267 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:10.936236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbbc39-21b3-4209-a02b-0fe12693b267 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:10.977954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186d718-c0f5-4880-8bdf-5c7af43fc890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:10.978792 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186d718-c0f5-4880-8bdf-5c7af43fc890 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:10.979078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186d718-c0f5-4880-8bdf-5c7af43fc890 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:10.979352 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186d718-c0f5-4880-8bdf-5c7af43fc890 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:10.994755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186d718-c0f5-4880-8bdf-5c7af43fc890 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:10.996338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186d718-c0f5-4880-8bdf-5c7af43fc890 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:11.018375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-1186d718-c0f5-4880-8bdf-5c7af43fc890 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:54:11.018375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:11.018375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:11.018375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:11.018375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:11.018375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:11.018375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:11.018375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:11.018375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:11.018375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:11.018375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:11.018375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:11.018375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:54:11.018375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:54:11.018375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:54:11.018375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:11.019805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:11.021345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:11.021345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:11.021345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:54:11.021345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:54:11.021345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:54:11.021345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:54:11.021345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:54:11.021345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:54:11.021345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:54:11.021345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:11.021345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 516/1034] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Jul 18 14:54:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:11.026710 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5513939-63a4-4795-be2a-4aafc7912ae4 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:11.091176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:11.133402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5513939-63a4-4795-be2a-4aafc7912ae4 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:11.181094 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1202a8b-8464-4470-ae5a-6125348f636d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:11.181769 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1202a8b-8464-4470-ae5a-6125348f636d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:11.182055 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1202a8b-8464-4470-ae5a-6125348f636d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:11.182332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1202a8b-8464-4470-ae5a-6125348f636d None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:11.183206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1202a8b-8464-4470-ae5a-6125348f636d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:11.183522 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1202a8b-8464-4470-ae5a-6125348f636d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:11.185057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1202a8b-8464-4470-ae5a-6125348f636d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:11.203829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 519/1035] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Jul 18 14:54:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:11.210065 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186d718-c0f5-4880-8bdf-5c7af43fc890 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:11.233741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:11.321176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186d718-c0f5-4880-8bdf-5c7af43fc890 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:11.356523 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4503e7-3073-490b-a59b-09108d833b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:11.357826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4503e7-3073-490b-a59b-09108d833b4f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:11.357826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4503e7-3073-490b-a59b-09108d833b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:11.357826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4503e7-3073-490b-a59b-09108d833b4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:11.429846 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4503e7-3073-490b-a59b-09108d833b4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:11.431350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4503e7-3073-490b-a59b-09108d833b4f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:11.454404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:11.539091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 517/1036] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:54:11 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:54:11.544895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1202a8b-8464-4470-ae5a-6125348f636d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:11.657890 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1202a8b-8464-4470-ae5a-6125348f636d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:11.682029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:11.694314 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dca339-fea1-4d50-8101-9bf9a0b73f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:11.695373 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dca339-fea1-4d50-8101-9bf9a0b73f2a None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:11.695835 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dca339-fea1-4d50-8101-9bf9a0b73f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:11.696269 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dca339-fea1-4d50-8101-9bf9a0b73f2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:11.706670 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dca339-fea1-4d50-8101-9bf9a0b73f2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:11.709174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dca339-fea1-4d50-8101-9bf9a0b73f2a None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:11.728420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-e1dca339-fea1-4d50-8101-9bf9a0b73f2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:54:11.728420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:11.728420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:11.728420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:11.728420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:11.728420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:11.728420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:11.728420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:11.728420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:11.728420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:11.728420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:11.728420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:11.728420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:54:11.728420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:54:11.728420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:11.729496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:11.730732 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:54:11.730732 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:54:11.730732 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:54:11.730732 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:11.734727 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 520/1037] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:11.736832 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4503e7-3073-490b-a59b-09108d833b4f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:11.841182 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4503e7-3073-490b-a59b-09108d833b4f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:11.861776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:11.876820 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d219dab-99df-4e6f-869e-c6aca6d61bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:11.877544 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d219dab-99df-4e6f-869e-c6aca6d61bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:11.877828 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d219dab-99df-4e6f-869e-c6aca6d61bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:11.878149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d219dab-99df-4e6f-869e-c6aca6d61bd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:11.878900 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d219dab-99df-4e6f-869e-c6aca6d61bd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:11.879204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d219dab-99df-4e6f-869e-c6aca6d61bd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:11.881155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d219dab-99df-4e6f-869e-c6aca6d61bd8 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:11.892723 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 518/1038] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:11.898403 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dca339-fea1-4d50-8101-9bf9a0b73f2a None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:12.019788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dca339-fea1-4d50-8101-9bf9a0b73f2a None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:12.055255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166ae08-b9d5-4bcc-9500-43ff16efef51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:12.056051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166ae08-b9d5-4bcc-9500-43ff16efef51 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:12.056330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166ae08-b9d5-4bcc-9500-43ff16efef51 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:12.056600 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166ae08-b9d5-4bcc-9500-43ff16efef51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:12.063506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166ae08-b9d5-4bcc-9500-43ff16efef51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:12.064948 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166ae08-b9d5-4bcc-9500-43ff16efef51 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:12.074108 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ad3b8-6723-4e51-a3cb-e5fe3cfbb91c None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:12.074401 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ad3b8-6723-4e51-a3cb-e5fe3cfbb91c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:12.074709 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ad3b8-6723-4e51-a3cb-e5fe3cfbb91c None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:12.076389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 519/1039] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:12.081214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-e166ae08-b9d5-4bcc-9500-43ff16efef51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:54:12.081214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:12.081214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:12.081214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:12.081214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:12.081214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:12.081214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:12.081214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:12.081214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:12.081214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:12.081214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:12.081214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:12.081214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:54:12.081214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:54:12.081214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:12.082167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:12.083309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:54:12.083309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:54:12.083309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:54:12.083309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:54:12.083309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:54:12.083309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:54:12.083309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:54:12.083309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:12.083935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb1ec7-023f-4bf7-b9f4-c890dad71286 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:12.083935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb1ec7-023f-4bf7-b9f4-c890dad71286 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:12.083935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb1ec7-023f-4bf7-b9f4-c890dad71286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:12.084186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 521/1040] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:54:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:54:12.089625 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166ae08-b9d5-4bcc-9500-43ff16efef51 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:12.105556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-dcbb1ec7-023f-4bf7-b9f4-c890dad71286 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:12.106995 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-dcbb1ec7-023f-4bf7-b9f4-c890dad71286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:12.128512 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:12.184097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 520/1041] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:12.200053 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166ae08-b9d5-4bcc-9500-43ff16efef51 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:12.209123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb1ec7-023f-4bf7-b9f4-c890dad71286 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:12.238565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558132d-8e1d-4eb3-a992-14ab9aef73af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:12.239908 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558132d-8e1d-4eb3-a992-14ab9aef73af None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:12.240217 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558132d-8e1d-4eb3-a992-14ab9aef73af None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:12.240492 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558132d-8e1d-4eb3-a992-14ab9aef73af None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:12.241458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558132d-8e1d-4eb3-a992-14ab9aef73af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:12.241904 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558132d-8e1d-4eb3-a992-14ab9aef73af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:12.243365 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558132d-8e1d-4eb3-a992-14ab9aef73af None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:12.261385 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 522/1042] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:54:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:12.262681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558132d-8e1d-4eb3-a992-14ab9aef73af None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:12.321227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb1ec7-023f-4bf7-b9f4-c890dad71286 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:12.333255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:12.359332 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d472ec-7f48-4c54-806f-596ba3fd7a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:12.360316 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d472ec-7f48-4c54-806f-596ba3fd7a3b None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:12.360605 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d472ec-7f48-4c54-806f-596ba3fd7a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:12.360904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d472ec-7f48-4c54-806f-596ba3fd7a3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:12.362207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d472ec-7f48-4c54-806f-596ba3fd7a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:12.364196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d472ec-7f48-4c54-806f-596ba3fd7a3b None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:12.370631 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 521/1043] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:12.372063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d472ec-7f48-4c54-806f-596ba3fd7a3b None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:12.373156 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:12.381605 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558132d-8e1d-4eb3-a992-14ab9aef73af None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:12.423194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1981b-3484-4841-bd4d-34d83d38f221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:12.423846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1981b-3484-4841-bd4d-34d83d38f221 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:12.424127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1981b-3484-4841-bd4d-34d83d38f221 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:12.424400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1981b-3484-4841-bd4d-34d83d38f221 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:12.425229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1981b-3484-4841-bd4d-34d83d38f221 None admin] RBAC: Adding query filter params (role.id=f7a8b3159cc4475ebac07df3543883cc, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=7e1355d18fb049468651b3675c217d60) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:12.425538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1981b-3484-4841-bd4d-34d83d38f221 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:12.427272 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1981b-3484-4841-bd4d-34d83d38f221 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:12.443193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 523/1044] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:54:12 2022] GET /identity/v3/role_assignments?user.id=7e1355d18fb049468651b3675c217d60&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=f7a8b3159cc4475ebac07df3543883cc => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:12.485371 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d472ec-7f48-4c54-806f-596ba3fd7a3b None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:12.525805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64101dd2-f416-4be4-8628-673632363375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:12.526502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64101dd2-f416-4be4-8628-673632363375 None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:12.526821 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64101dd2-f416-4be4-8628-673632363375 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:12.527102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64101dd2-f416-4be4-8628-673632363375 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:12.527833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64101dd2-f416-4be4-8628-673632363375 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:12.529439 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64101dd2-f416-4be4-8628-673632363375 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:12.566442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5551b5-d12f-4c37-ad7c-276f6bdaee2c None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:12.567085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5551b5-d12f-4c37-ad7c-276f6bdaee2c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:12.567085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5551b5-d12f-4c37-ad7c-276f6bdaee2c None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:12.569457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 524/1045] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:54:12.575188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2578db-c419-44fc-af44-68b73671eb4a None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:12.575484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2578db-c419-44fc-af44-68b73671eb4a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:12.575770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2578db-c419-44fc-af44-68b73671eb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:12.594262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-8f2578db-c419-44fc-af44-68b73671eb4a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:12.595523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-8f2578db-c419-44fc-af44-68b73671eb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:12.654718 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 522/1046] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:54:12 2022] POST /identity/v3/users => generated 285 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:54:12.668234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 525/1047] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:54:12.693311 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64101dd2-f416-4be4-8628-673632363375 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:12.733496 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:12.811023 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64101dd2-f416-4be4-8628-673632363375 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:12.853626 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a96a8-4d67-4c1f-8ce4-b6d509c16008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:12.854302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a96a8-4d67-4c1f-8ce4-b6d509c16008 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:12.854618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a96a8-4d67-4c1f-8ce4-b6d509c16008 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:12.854906 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a96a8-4d67-4c1f-8ce4-b6d509c16008 None admin] PATH_INFO: `/v3/services/image` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:12.855836 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a96a8-4d67-4c1f-8ce4-b6d509c16008 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:12.857289 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a96a8-4d67-4c1f-8ce4-b6d509c16008 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-e49a96a8-4d67-4c1f-8ce4-b6d509c16008 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:12.869618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:12.870921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:12.870921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:12.870921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:12.870921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:12.870921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:12.870921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:12.870921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:54:12.870921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:54:12.870921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:54:12.870921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:54:12.870921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:12.870921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:12.870921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:12.870921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:12.870921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:12.870921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:12.872010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:12.872010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:12.872010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:12.872010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:12.872010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:12.872010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:12.872010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:12.872010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:12.872010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:12.872010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:12.872010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:54:12.872010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:54:12.872010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:54:12.872010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:12.872010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 523/1048] 10.222.0.83 () {56 vars in 1105 bytes} [Mon Jul 18 14:54:12 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:12.878572 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2578db-c419-44fc-af44-68b73671eb4a None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:12.979322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:12.996384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2578db-c419-44fc-af44-68b73671eb4a None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:13.032811 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e116e-1821-411e-94a0-8baff3886ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:13.033791 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e116e-1821-411e-94a0-8baff3886ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:13.034228 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e116e-1821-411e-94a0-8baff3886ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:13.034684 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e116e-1821-411e-94a0-8baff3886ce7 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:13.035593 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e116e-1821-411e-94a0-8baff3886ce7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:13.037204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e116e-1821-411e-94a0-8baff3886ce7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:13.038783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e116e-1821-411e-94a0-8baff3886ce7 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:13.044300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 526/1049] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:13.050738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a96a8-4d67-4c1f-8ce4-b6d509c16008 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:13.153055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:13.157564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a96a8-4d67-4c1f-8ce4-b6d509c16008 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:13.195795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ade082-68f9-4651-8ea3-e586a88c53c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:13.196486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ade082-68f9-4651-8ea3-e586a88c53c4 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:13.196788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ade082-68f9-4651-8ea3-e586a88c53c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:13.197047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ade082-68f9-4651-8ea3-e586a88c53c4 None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:13.197757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ade082-68f9-4651-8ea3-e586a88c53c4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:13.198063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ade082-68f9-4651-8ea3-e586a88c53c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:13.199482 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ade082-68f9-4651-8ea3-e586a88c53c4 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:13.204995 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 524/1050] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:13.210685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e116e-1821-411e-94a0-8baff3886ce7 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:13.369909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e116e-1821-411e-94a0-8baff3886ce7 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:13.405105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a34e1b-47f3-4c26-8241-330365fb15e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:13.405773 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a34e1b-47f3-4c26-8241-330365fb15e3 None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:13.406055 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a34e1b-47f3-4c26-8241-330365fb15e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:13.406326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a34e1b-47f3-4c26-8241-330365fb15e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:13.407103 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a34e1b-47f3-4c26-8241-330365fb15e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:13.408474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a34e1b-47f3-4c26-8241-330365fb15e3 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:13.533372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 527/1051] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Jul 18 14:54:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:54:14.502728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480af97-435e-457f-87e5-7ba4adf6ce61 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:14.503348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480af97-435e-457f-87e5-7ba4adf6ce61 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:14.503348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480af97-435e-457f-87e5-7ba4adf6ce61 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:14.505095 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 525/1052] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:14.510196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdc7fb-39aa-4fc9-a867-3728f835d574 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:14.510639 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdc7fb-39aa-4fc9-a867-3728f835d574 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:14.510810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdc7fb-39aa-4fc9-a867-3728f835d574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:14.528546 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-abfdc7fb-39aa-4fc9-a867-3728f835d574 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:14.529742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-abfdc7fb-39aa-4fc9-a867-3728f835d574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:14.531829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:14.615407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 528/1053] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:14.637646 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480af97-435e-457f-87e5-7ba4adf6ce61 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:14.750198 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480af97-435e-457f-87e5-7ba4adf6ce61 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:14.785596 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fe7dd-1a7a-49aa-af4d-fa12b19c2ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:14.786979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fe7dd-1a7a-49aa-af4d-fa12b19c2ecb None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:14.787264 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fe7dd-1a7a-49aa-af4d-fa12b19c2ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:14.787554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fe7dd-1a7a-49aa-af4d-fa12b19c2ecb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:14.797887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1fe7dd-1a7a-49aa-af4d-fa12b19c2ecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:54:14.806477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-3b1fe7dd-1a7a-49aa-af4d-fa12b19c2ecb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:54:14.806477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:14.806477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:14.806477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:14.806477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:14.806477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:14.806477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:14.806477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:14.806477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:14.806477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:14.806477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:14.806477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:14.806477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:54:14.806477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:54:14.806477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:54:14.806477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:54:14.807568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:54:14.807568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:54:14.807568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:54:14.807568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:54:14.807568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:14.807568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:14.807568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:14.807568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:14.807568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:14.807568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:14.807568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:14.807568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:14.807568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:14.807568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:14.807568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:14.807568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:14.808438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:14.808438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:14.808438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:14.808438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:14.808438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:54:14.808438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:54:14.808438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:54:14.808438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:54:14.808438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:54:14.808438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:54:14.808438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:54:14.808438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:14.808438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 526/1054] 10.222.0.83 () {56 vars in 1120 bytes} [Mon Jul 18 14:54:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:14.815184 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfdc7fb-39aa-4fc9-a867-3728f835d574 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:14.856760 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4115a45-5346-4016-8152-5d9d1ef74ab2 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:14.857056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4115a45-5346-4016-8152-5d9d1ef74ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:14.857340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4115a45-5346-4016-8152-5d9d1ef74ab2 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:14.858756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 527/1055] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:54:14.863638 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390f459-f132-44f4-b08d-4bbe7ba7fe5e None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:14.864070 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390f459-f132-44f4-b08d-4bbe7ba7fe5e None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:14.864335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390f459-f132-44f4-b08d-4bbe7ba7fe5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:14.882901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-7390f459-f132-44f4-b08d-4bbe7ba7fe5e None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:14.884461 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7390f459-f132-44f4-b08d-4bbe7ba7fe5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:14.904524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:14.933055 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfdc7fb-39aa-4fc9-a867-3728f835d574 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:14.965504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 528/1056] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:54:14.978297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a783c-2735-471a-9a55-350c143227a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:14.979009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a783c-2735-471a-9a55-350c143227a9 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:14.979295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a783c-2735-471a-9a55-350c143227a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:14.979567 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a783c-2735-471a-9a55-350c143227a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:14.980573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a783c-2735-471a-9a55-350c143227a9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:14.983930 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a783c-2735-471a-9a55-350c143227a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:14.983930 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a783c-2735-471a-9a55-350c143227a9 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:14.988969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:14.993251 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390f459-f132-44f4-b08d-4bbe7ba7fe5e None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:14.994704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 529/1057] 10.222.0.83 () {56 vars in 1115 bytes} [Mon Jul 18 14:54:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:14.999193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a783c-2735-471a-9a55-350c143227a9 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:15.106684 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a783c-2735-471a-9a55-350c143227a9 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:15.106896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390f459-f132-44f4-b08d-4bbe7ba7fe5e None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:15.142774 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24969df8-c0ed-4236-a7ba-b9f504c82a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:15.143603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24969df8-c0ed-4236-a7ba-b9f504c82a57 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:15.143900 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24969df8-c0ed-4236-a7ba-b9f504c82a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:15.144173 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24969df8-c0ed-4236-a7ba-b9f504c82a57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:15.145403 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9180b-9e3a-4a23-a6a7-12cd1fe41be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:15.146378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9180b-9e3a-4a23-a6a7-12cd1fe41be0 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:15.146696 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9180b-9e3a-4a23-a6a7-12cd1fe41be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:15.147135 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9180b-9e3a-4a23-a6a7-12cd1fe41be0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:15.155142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24969df8-c0ed-4236-a7ba-b9f504c82a57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:54:15.158410 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9180b-9e3a-4a23-a6a7-12cd1fe41be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:15.159914 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9180b-9e3a-4a23-a6a7-12cd1fe41be0 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:15.163938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-24969df8-c0ed-4236-a7ba-b9f504c82a57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:54:15.163938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:15.163938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:15.163938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:15.163938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:15.163938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:15.163938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:15.163938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:15.163938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:15.163938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:15.163938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:15.163938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:15.163938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:54:15.163938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:54:15.163938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:54:15.163938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:54:15.165138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:54:15.165138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:54:15.165138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:54:15.165138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:54:15.165138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:15.165138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:15.165138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:15.165138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:15.165138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:15.165138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:15.165138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:15.165138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:15.165138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:15.165138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:15.165138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:15.165138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:15.166059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:15.166059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:15.166059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:15.166059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:15.166059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:54:15.166059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:54:15.166059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:54:15.166059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:54:15.166059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:54:15.166059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:54:15.166059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:54:15.166059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:15.166059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 529/1058] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:54:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:54:15.171428 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24969df8-c0ed-4236-a7ba-b9f504c82a57 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:15.185313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-e1a9180b-9e3a-4a23-a6a7-12cd1fe41be0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:54:15.185313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:15.185313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:15.185313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:15.185313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:15.185313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:15.185313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:15.185313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:15.185313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:15.185313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:15.185313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:15.185313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:15.185313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:54:15.185313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:54:15.185313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:15.186324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:15.187651 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:54:15.187651 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:54:15.187651 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:54:15.187651 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:15.187651 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 530/1059] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:54:15.188936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:15.192859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9180b-9e3a-4a23-a6a7-12cd1fe41be0 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:15.246121 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:15.295330 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24969df8-c0ed-4236-a7ba-b9f504c82a57 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:15.304738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9180b-9e3a-4a23-a6a7-12cd1fe41be0 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:15.341420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3678963-d207-4401-953f-753dbd67615c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:15.342329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3678963-d207-4401-953f-753dbd67615c None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:15.342934 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3678963-d207-4401-953f-753dbd67615c None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:15.342934 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3678963-d207-4401-953f-753dbd67615c None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:15.343479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3678963-d207-4401-953f-753dbd67615c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:15.343789 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3678963-d207-4401-953f-753dbd67615c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:15.345189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3678963-d207-4401-953f-753dbd67615c None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:15.354602 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 530/1060] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:54:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:15.360574 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3678963-d207-4401-953f-753dbd67615c None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:15.399767 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944920a2-aff5-485c-8c89-572867d24d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:15.400742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944920a2-aff5-485c-8c89-572867d24d00 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:15.401197 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944920a2-aff5-485c-8c89-572867d24d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:15.401648 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944920a2-aff5-485c-8c89-572867d24d00 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:15.402555 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944920a2-aff5-485c-8c89-572867d24d00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:15.403031 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944920a2-aff5-485c-8c89-572867d24d00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:15.404416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944920a2-aff5-485c-8c89-572867d24d00 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:15.423306 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 531/1061] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:15.424641 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944920a2-aff5-485c-8c89-572867d24d00 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:15.452254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:15.480676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3678963-d207-4401-953f-753dbd67615c None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:15.517699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034e4e2-2542-4934-8d25-69cc2a4ee89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:15.518610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034e4e2-2542-4934-8d25-69cc2a4ee89c None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:15.518957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034e4e2-2542-4934-8d25-69cc2a4ee89c None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:15.519235 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034e4e2-2542-4934-8d25-69cc2a4ee89c None admin] PATH_INFO: `/v3/users/dc86af4cb7e64ccab2ad612d419ff507` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:15.536881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034e4e2-2542-4934-8d25-69cc2a4ee89c None admin] RBAC: Authorizing `identity:get_user(user_id=dc86af4cb7e64ccab2ad612d419ff507)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:15.538384 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034e4e2-2542-4934-8d25-69cc2a4ee89c None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:15.544982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 531/1062] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:54:15 2022] GET /identity/v3/users/dc86af4cb7e64ccab2ad612d419ff507 => generated 285 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:15.545161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944920a2-aff5-485c-8c89-572867d24d00 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:15.547454 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034e4e2-2542-4934-8d25-69cc2a4ee89c None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:15.559147 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:15.585518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151024a7-3b2e-459b-bd2b-6b13dd4ef182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:15.586342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151024a7-3b2e-459b-bd2b-6b13dd4ef182 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:15.586702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151024a7-3b2e-459b-bd2b-6b13dd4ef182 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:15.586981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151024a7-3b2e-459b-bd2b-6b13dd4ef182 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:15.594371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151024a7-3b2e-459b-bd2b-6b13dd4ef182 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:15.596056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151024a7-3b2e-459b-bd2b-6b13dd4ef182 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:15.613520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-151024a7-3b2e-459b-bd2b-6b13dd4ef182 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:54:15.613520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:15.613520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:15.613520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:15.613520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:15.613520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:15.613520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:15.613520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:15.613520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:15.613520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:15.613520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:15.613520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:15.613520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:54:15.613520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:54:15.613520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:54:15.613520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:15.615062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:15.616161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:15.616161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:15.616161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:54:15.616161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:54:15.616161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:54:15.616161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:54:15.616161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:54:15.616161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:54:15.616161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:54:15.616161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:15.616161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 532/1063] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Jul 18 14:54:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:54:15.620747 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151024a7-3b2e-459b-bd2b-6b13dd4ef182 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:15.672864 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034e4e2-2542-4934-8d25-69cc2a4ee89c None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:15.704089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:15.710086 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5dd44d-3f63-4fbe-a3c3-0bb71738885a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:15.710779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dd44d-3f63-4fbe-a3c3-0bb71738885a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:15.711068 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dd44d-3f63-4fbe-a3c3-0bb71738885a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:15.711357 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dd44d-3f63-4fbe-a3c3-0bb71738885a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:15.728740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151024a7-3b2e-459b-bd2b-6b13dd4ef182 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:15.765214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d29e50-a735-43a2-91d2-d96351f8f853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:15.765887 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d29e50-a735-43a2-91d2-d96351f8f853 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:15.766166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d29e50-a735-43a2-91d2-d96351f8f853 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:15.766474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d29e50-a735-43a2-91d2-d96351f8f853 None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:15.767330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d29e50-a735-43a2-91d2-d96351f8f853 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:15.767639 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d29e50-a735-43a2-91d2-d96351f8f853 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:15.769079 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d29e50-a735-43a2-91d2-d96351f8f853 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:15.781357 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5dd44d-3f63-4fbe-a3c3-0bb71738885a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:15.782914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5dd44d-3f63-4fbe-a3c3-0bb71738885a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:15.787524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 533/1064] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Jul 18 14:54:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:15.788832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b1998-e95e-4af4-9cc3-9f0589c4cfae None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:15.789121 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b1998-e95e-4af4-9cc3-9f0589c4cfae None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:15.789399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b1998-e95e-4af4-9cc3-9f0589c4cfae None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:15.790643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 534/1065] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:15.793302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b1998-e95e-4af4-9cc3-9f0589c4cfae None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:15.822924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:15.888555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 532/1066] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:54:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 14:54:15.889773 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28242e35-f5e2-4a70-ab52-f3de7ceb3c69 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:15.890070 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28242e35-f5e2-4a70-ab52-f3de7ceb3c69 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:15.890345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28242e35-f5e2-4a70-ab52-f3de7ceb3c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:15.900456 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b1998-e95e-4af4-9cc3-9f0589c4cfae None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:15.908276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-28242e35-f5e2-4a70-ab52-f3de7ceb3c69 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:15.909503 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-28242e35-f5e2-4a70-ab52-f3de7ceb3c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:15.936599 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dba2bb-b2ad-4b26-bdc1-82042e6ea6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:15.937403 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dba2bb-b2ad-4b26-bdc1-82042e6ea6ae None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:15.937687 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dba2bb-b2ad-4b26-bdc1-82042e6ea6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:15.937969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dba2bb-b2ad-4b26-bdc1-82042e6ea6ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:15.939402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:15.948514 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dba2bb-b2ad-4b26-bdc1-82042e6ea6ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:15.955286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dba2bb-b2ad-4b26-bdc1-82042e6ea6ae None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:15.967816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-85dba2bb-b2ad-4b26-bdc1-82042e6ea6ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:54:15.967816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:15.967816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:15.967816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:15.967816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:15.967816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:15.967816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:15.967816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:15.967816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:15.967816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:15.967816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:15.967816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:15.967816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:54:15.967816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:54:15.967816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:15.969000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:15.970280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:54:15.970280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:54:15.970280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:54:15.970280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:15.970280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 535/1067] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:54:15.970280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dba2bb-b2ad-4b26-bdc1-82042e6ea6ae None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:15.991663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 533/1068] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:15.992361 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28242e35-f5e2-4a70-ab52-f3de7ceb3c69 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:16.059022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:16.080141 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dba2bb-b2ad-4b26-bdc1-82042e6ea6ae None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:16.101863 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28242e35-f5e2-4a70-ab52-f3de7ceb3c69 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:16.116066 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3410aa7-a85f-4a74-bbad-31d734c59fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:16.117029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3410aa7-a85f-4a74-bbad-31d734c59fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:16.117316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3410aa7-a85f-4a74-bbad-31d734c59fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:16.117592 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3410aa7-a85f-4a74-bbad-31d734c59fb2 None admin] PATH_INFO: `/v3/projects/3f57219ec685451cb2153ec3a41dbdca` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:16.118809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3410aa7-a85f-4a74-bbad-31d734c59fb2 None admin] RBAC: Authorizing `identity:get_project(project_id=3f57219ec685451cb2153ec3a41dbdca)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:16.120262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3410aa7-a85f-4a74-bbad-31d734c59fb2 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:16.121978 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 536/1069] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:54:15 2022] GET /identity/v3/projects/3f57219ec685451cb2153ec3a41dbdca => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:16.122815 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3410aa7-a85f-4a74-bbad-31d734c59fb2 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:16.136917 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c689778-bd76-4073-b875-ca0903b94a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:16.137588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c689778-bd76-4073-b875-ca0903b94a75 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:16.137876 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c689778-bd76-4073-b875-ca0903b94a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:16.138160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c689778-bd76-4073-b875-ca0903b94a75 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:16.138974 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c689778-bd76-4073-b875-ca0903b94a75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:16.139227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c689778-bd76-4073-b875-ca0903b94a75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:16.140633 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c689778-bd76-4073-b875-ca0903b94a75 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:16.152625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 534/1070] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:16.153350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c689778-bd76-4073-b875-ca0903b94a75 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:16.179349 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:16.227337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3410aa7-a85f-4a74-bbad-31d734c59fb2 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:16.255572 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c689778-bd76-4073-b875-ca0903b94a75 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:16.263019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ede443-ed0f-4f35-91e5-a48b1e769668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:16.263665 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ede443-ed0f-4f35-91e5-a48b1e769668 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:16.263947 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ede443-ed0f-4f35-91e5-a48b1e769668 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:16.264220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ede443-ed0f-4f35-91e5-a48b1e769668 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:16.280467 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:16.291803 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9403ee-9a4a-4938-97cb-728030fc4aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:16.292679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9403ee-9a4a-4938-97cb-728030fc4aee None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:16.293073 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9403ee-9a4a-4938-97cb-728030fc4aee None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:16.293417 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9403ee-9a4a-4938-97cb-728030fc4aee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:16.294450 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9403ee-9a4a-4938-97cb-728030fc4aee None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=3f57219ec685451cb2153ec3a41dbdca, user.id=dc86af4cb7e64ccab2ad612d419ff507) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:16.295000 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9403ee-9a4a-4938-97cb-728030fc4aee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:16.296696 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9403ee-9a4a-4938-97cb-728030fc4aee None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:16.303422 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 535/1071] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:54:16 2022] GET /identity/v3/role_assignments?user.id=dc86af4cb7e64ccab2ad612d419ff507&scope.project.id=3f57219ec685451cb2153ec3a41dbdca&role.id=5891920310b54787bd1152d736d2a3f1 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:16.304231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9403ee-9a4a-4938-97cb-728030fc4aee None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:16.345940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ede443-ed0f-4f35-91e5-a48b1e769668 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:16.348883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ede443-ed0f-4f35-91e5-a48b1e769668 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:16.418873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:16.435555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9403ee-9a4a-4938-97cb-728030fc4aee None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:16.475232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 537/1072] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:54:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:54:16.481772 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ede443-ed0f-4f35-91e5-a48b1e769668 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:16.486403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b937b8-101b-49e4-b632-e206a7788e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:16.487435 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b937b8-101b-49e4-b632-e206a7788e78 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:16.487734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b937b8-101b-49e4-b632-e206a7788e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:16.487996 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b937b8-101b-49e4-b632-e206a7788e78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:16.496580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b937b8-101b-49e4-b632-e206a7788e78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:16.498185 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b937b8-101b-49e4-b632-e206a7788e78 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:16.514348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-03b937b8-101b-49e4-b632-e206a7788e78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:54:16.514348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:16.514348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:16.514348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:16.514348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:16.514348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:16.514348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:16.514348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:16.514348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:16.514348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:16.514348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:16.514348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:16.514348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:54:16.514348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:54:16.514348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:16.515438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:16.516366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:54:16.516366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:54:16.516366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:54:16.516366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:54:16.516366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:54:16.516366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:54:16.516366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:54:16.516366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:16.516366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 536/1073] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:54:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:16.522143 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b937b8-101b-49e4-b632-e206a7788e78 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:16.537041 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:16.603792 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ede443-ed0f-4f35-91e5-a48b1e769668 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:16.644412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6819f2-5854-480a-8c70-269f1b67db15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:16.645975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6819f2-5854-480a-8c70-269f1b67db15 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:16.646282 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6819f2-5854-480a-8c70-269f1b67db15 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:16.646554 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6819f2-5854-480a-8c70-269f1b67db15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:16.652568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b937b8-101b-49e4-b632-e206a7788e78 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:16.657687 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6819f2-5854-480a-8c70-269f1b67db15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:16.659253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6819f2-5854-480a-8c70-269f1b67db15 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:16.681616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-dd6819f2-5854-480a-8c70-269f1b67db15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:54:16.681616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:16.681616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:16.681616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:16.681616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:16.681616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:16.681616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:16.681616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:16.681616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:16.681616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:16.681616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:16.681616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:16.681616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:54:16.681616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:54:16.681616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:16.683270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:16.686458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:54:16.686458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:54:16.686458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:54:16.686458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:16.686458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 538/1074] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:54:16.690900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6819f2-5854-480a-8c70-269f1b67db15 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:16.699915 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567fc3c7-ae53-4309-ab62-d6de6051f5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:16.701031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fc3c7-ae53-4309-ab62-d6de6051f5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:16.701338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fc3c7-ae53-4309-ab62-d6de6051f5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:16.701737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fc3c7-ae53-4309-ab62-d6de6051f5e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:16.702571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567fc3c7-ae53-4309-ab62-d6de6051f5e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:16.703263 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567fc3c7-ae53-4309-ab62-d6de6051f5e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:16.705144 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567fc3c7-ae53-4309-ab62-d6de6051f5e6 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:16.721492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 537/1075] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:54:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:16.728061 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567fc3c7-ae53-4309-ab62-d6de6051f5e6 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:16.740678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:16.809589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:16.813608 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6819f2-5854-480a-8c70-269f1b67db15 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:16.853593 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1928d7-dc69-481c-bd1b-e47e5d67be1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:16.854388 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1928d7-dc69-481c-bd1b-e47e5d67be1c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:16.854715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1928d7-dc69-481c-bd1b-e47e5d67be1c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:16.855259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1928d7-dc69-481c-bd1b-e47e5d67be1c None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:16.856001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1928d7-dc69-481c-bd1b-e47e5d67be1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:16.856319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1928d7-dc69-481c-bd1b-e47e5d67be1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:16.857719 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1928d7-dc69-481c-bd1b-e47e5d67be1c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:16.865640 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567fc3c7-ae53-4309-ab62-d6de6051f5e6 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:16.873335 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 539/1076] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:16.916880 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd337e-8963-4d6d-9f0d-1da4247b18b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:16.917908 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd337e-8963-4d6d-9f0d-1da4247b18b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:16.918194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd337e-8963-4d6d-9f0d-1da4247b18b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:16.918466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd337e-8963-4d6d-9f0d-1da4247b18b5 None admin] PATH_INFO: `/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/7e1355d18fb049468651b3675c217d60/roles/f7a8b3159cc4475ebac07df3543883cc` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:16.920886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd337e-8963-4d6d-9f0d-1da4247b18b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f777cd6a0c744e594f8e29ab58cf8bd, user_id=7e1355d18fb049468651b3675c217d60, role_id=f7a8b3159cc4475ebac07df3543883cc)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:16.924731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd337e-8963-4d6d-9f0d-1da4247b18b5 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:16.997875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 538/1077] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:54:16 2022] PUT /identity/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/7e1355d18fb049468651b3675c217d60/roles/f7a8b3159cc4475ebac07df3543883cc => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:54:18.386521 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6766f-dd07-46d4-922f-adfd279d938b None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:18.387135 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6766f-dd07-46d4-922f-adfd279d938b None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:18.387231 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6766f-dd07-46d4-922f-adfd279d938b None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:18.388658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 540/1078] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:18.394003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169b575-7ab9-4ffb-a304-eb5ab61d428b None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:18.394442 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169b575-7ab9-4ffb-a304-eb5ab61d428b None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:18.394773 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169b575-7ab9-4ffb-a304-eb5ab61d428b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:18.411989 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-0169b575-7ab9-4ffb-a304-eb5ab61d428b None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:18.413186 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-0169b575-7ab9-4ffb-a304-eb5ab61d428b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:18.473748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:18.481978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 539/1079] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:18.507152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6766f-dd07-46d4-922f-adfd279d938b None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:18.608848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6766f-dd07-46d4-922f-adfd279d938b None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:18.642777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2c301-70f6-4f5b-a6be-7044a2b8cd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:18.643484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2c301-70f6-4f5b-a6be-7044a2b8cd38 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:18.643763 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2c301-70f6-4f5b-a6be-7044a2b8cd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:18.644034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2c301-70f6-4f5b-a6be-7044a2b8cd38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:18.654403 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2c301-70f6-4f5b-a6be-7044a2b8cd38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:54:18.662838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-8aa2c301-70f6-4f5b-a6be-7044a2b8cd38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:54:18.662838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:18.662838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:18.662838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:18.662838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:18.662838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:18.662838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:18.662838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:18.662838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:18.662838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:18.662838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:18.662838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:18.662838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:54:18.662838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:54:18.662838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:54:18.662838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:54:18.663894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:54:18.663894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:54:18.663894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:54:18.663894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:54:18.663894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:18.663894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:18.663894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:18.663894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:18.663894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:18.663894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:18.663894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:18.663894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:18.663894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:18.663894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:18.663894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:18.663894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:18.664768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:18.664768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:18.664768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:18.664768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:18.664768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:54:18.664768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:54:18.664768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:54:18.664768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:54:18.664768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:54:18.664768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:54:18.664768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:54:18.664768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:18.664768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 541/1080] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:54:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:18.670358 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169b575-7ab9-4ffb-a304-eb5ab61d428b None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:18.770675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169b575-7ab9-4ffb-a304-eb5ab61d428b None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:18.805301 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c020a33-3fe3-4339-9397-a205471631a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:18.805973 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c020a33-3fe3-4339-9397-a205471631a5 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:18.806258 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c020a33-3fe3-4339-9397-a205471631a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:18.806566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c020a33-3fe3-4339-9397-a205471631a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:18.807303 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c020a33-3fe3-4339-9397-a205471631a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:18.807613 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c020a33-3fe3-4339-9397-a205471631a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:18.808998 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c020a33-3fe3-4339-9397-a205471631a5 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:18.818214 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 540/1081] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:54:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:18.823888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2c301-70f6-4f5b-a6be-7044a2b8cd38 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:18.844210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:18.927895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2c301-70f6-4f5b-a6be-7044a2b8cd38 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:18.962171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefb1ec-d287-4b27-b0b6-7d0d0c64b338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:18.963101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefb1ec-d287-4b27-b0b6-7d0d0c64b338 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:18.963393 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefb1ec-d287-4b27-b0b6-7d0d0c64b338 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:18.963675 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefb1ec-d287-4b27-b0b6-7d0d0c64b338 None admin] PATH_INFO: `/v3/users/dc86af4cb7e64ccab2ad612d419ff507` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:18.964868 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefb1ec-d287-4b27-b0b6-7d0d0c64b338 None admin] RBAC: Authorizing `identity:get_user(user_id=dc86af4cb7e64ccab2ad612d419ff507)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:18.966309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefb1ec-d287-4b27-b0b6-7d0d0c64b338 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:18.968088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 542/1082] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:54:18 2022] GET /identity/v3/users/dc86af4cb7e64ccab2ad612d419ff507 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:18.973935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c020a33-3fe3-4339-9397-a205471631a5 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:19.033649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:19.077338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c020a33-3fe3-4339-9397-a205471631a5 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:19.111627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed5689-5aea-4566-be8b-123d6dc25f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:19.112505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed5689-5aea-4566-be8b-123d6dc25f79 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:19.112788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed5689-5aea-4566-be8b-123d6dc25f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:19.113065 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed5689-5aea-4566-be8b-123d6dc25f79 None admin] PATH_INFO: `/v3/projects/3f57219ec685451cb2153ec3a41dbdca` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:19.114456 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed5689-5aea-4566-be8b-123d6dc25f79 None admin] RBAC: Authorizing `identity:get_project(project_id=3f57219ec685451cb2153ec3a41dbdca)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:19.115935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed5689-5aea-4566-be8b-123d6dc25f79 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:19.117663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 541/1083] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:54:18 2022] GET /identity/v3/projects/3f57219ec685451cb2153ec3a41dbdca => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:19.123904 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefb1ec-d287-4b27-b0b6-7d0d0c64b338 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:19.211203 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:19.224466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f37e45-585a-4ba5-8b30-6ca724c7ca35 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:19.227874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f37e45-585a-4ba5-8b30-6ca724c7ca35 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:19.227874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f37e45-585a-4ba5-8b30-6ca724c7ca35 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:19.228061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefb1ec-d287-4b27-b0b6-7d0d0c64b338 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:19.228150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 542/1084] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:54:19.232494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5776a6-c809-4df5-904b-9405d1258093 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:19.232781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5776a6-c809-4df5-904b-9405d1258093 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:19.233062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5776a6-c809-4df5-904b-9405d1258093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:19.249986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-2f5776a6-c809-4df5-904b-9405d1258093 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:19.251230 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-2f5776a6-c809-4df5-904b-9405d1258093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:19.267140 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ade2f-b021-4647-9798-d01f43c08df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:19.268031 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ade2f-b021-4647-9798-d01f43c08df3 None admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:19.268322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ade2f-b021-4647-9798-d01f43c08df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:19.268604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ade2f-b021-4647-9798-d01f43c08df3 None admin] PATH_INFO: `/v3/projects/3f57219ec685451cb2153ec3a41dbdca/users/dc86af4cb7e64ccab2ad612d419ff507/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:19.271449 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ade2f-b021-4647-9798-d01f43c08df3 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f57219ec685451cb2153ec3a41dbdca, user_id=dc86af4cb7e64ccab2ad612d419ff507, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:19.272882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ade2f-b021-4647-9798-d01f43c08df3 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:19.321636 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 543/1085] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:54:19.346145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5776a6-c809-4df5-904b-9405d1258093 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:19.387482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 543/1086] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:54:19 2022] PUT /identity/v3/projects/3f57219ec685451cb2153ec3a41dbdca/users/dc86af4cb7e64ccab2ad612d419ff507/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:54:19.410870 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:19.468189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5776a6-c809-4df5-904b-9405d1258093 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:19.514431 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59184d8-0dad-4765-89b1-3f264e064e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:19.515690 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59184d8-0dad-4765-89b1-3f264e064e96 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:19.516194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59184d8-0dad-4765-89b1-3f264e064e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:19.516686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59184d8-0dad-4765-89b1-3f264e064e96 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:19.529679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59184d8-0dad-4765-89b1-3f264e064e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:54:19.539076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-f59184d8-0dad-4765-89b1-3f264e064e96 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:54:19.539076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:19.539076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:19.539076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:19.539076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:19.539076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:19.539076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:19.539076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:19.539076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:19.539076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:19.539076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:19.539076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:19.539076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:54:19.539076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:54:19.539076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:54:19.539076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:54:19.540128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:54:19.540128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:54:19.540128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:54:19.540128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:54:19.540128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:19.540128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:19.540128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:19.540128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:19.540128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:19.540128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:19.540128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:19.540128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:19.540128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:19.540128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:19.540128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:19.540128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:19.541013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:19.541013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:19.541013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:19.541013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:19.541013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:54:19.541013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:54:19.541013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:54:19.541013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:54:19.541013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:54:19.541013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:54:19.541013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:54:19.541013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:19.543881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 544/1087] 10.222.0.83 () {56 vars in 1120 bytes} [Mon Jul 18 14:54:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:54:19.550101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ade2f-b021-4647-9798-d01f43c08df3 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:19.670361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ade2f-b021-4647-9798-d01f43c08df3 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:19.707323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20ebf0-ba33-4aab-9f97-ceeb7ce30523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:19.708008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20ebf0-ba33-4aab-9f97-ceeb7ce30523 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:19.708300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20ebf0-ba33-4aab-9f97-ceeb7ce30523 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:19.708584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20ebf0-ba33-4aab-9f97-ceeb7ce30523 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:19.709348 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd20ebf0-ba33-4aab-9f97-ceeb7ce30523 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:19.709664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd20ebf0-ba33-4aab-9f97-ceeb7ce30523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:19.711089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd20ebf0-ba33-4aab-9f97-ceeb7ce30523 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:19.722664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 544/1088] 10.222.0.83 () {56 vars in 1115 bytes} [Mon Jul 18 14:54:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:19.726950 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59184d8-0dad-4765-89b1-3f264e064e96 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:19.838772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59184d8-0dad-4765-89b1-3f264e064e96 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:19.855753 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:19.874240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a8f54-3e6e-4019-a5df-c119c13eef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:19.875065 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a8f54-3e6e-4019-a5df-c119c13eef00 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:19.875349 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a8f54-3e6e-4019-a5df-c119c13eef00 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:19.875622 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a8f54-3e6e-4019-a5df-c119c13eef00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:19.886240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a8f54-3e6e-4019-a5df-c119c13eef00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:19.887718 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a8f54-3e6e-4019-a5df-c119c13eef00 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:19.905600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-d78a8f54-3e6e-4019-a5df-c119c13eef00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:54:19.905600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:19.905600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:19.905600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:19.905600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:19.905600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:19.905600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:19.905600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:19.905600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:19.905600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:19.905600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:19.905600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:19.905600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:54:19.905600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:54:19.905600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:19.906781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:19.908013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:54:19.908013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:54:19.908013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:54:19.908013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:19.911101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 545/1089] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:19.913176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20ebf0-ba33-4aab-9f97-ceeb7ce30523 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:19.925216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:20.015196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20ebf0-ba33-4aab-9f97-ceeb7ce30523 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:20.048819 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955815da-37df-456b-af8c-4bf369214232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:20.049488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955815da-37df-456b-af8c-4bf369214232 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:20.049773 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955815da-37df-456b-af8c-4bf369214232 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:20.050050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955815da-37df-456b-af8c-4bf369214232 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:20.050985 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955815da-37df-456b-af8c-4bf369214232 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:20.052049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955815da-37df-456b-af8c-4bf369214232 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:20.054552 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955815da-37df-456b-af8c-4bf369214232 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:20.066563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 545/1090] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:20.072459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a8f54-3e6e-4019-a5df-c119c13eef00 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:20.174451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a8f54-3e6e-4019-a5df-c119c13eef00 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:20.208213 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c894f0-c4ba-435f-ba2a-bb7259781b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:20.208998 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c894f0-c4ba-435f-ba2a-bb7259781b2a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:20.209283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c894f0-c4ba-435f-ba2a-bb7259781b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:20.209563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c894f0-c4ba-435f-ba2a-bb7259781b2a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:20.217028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c894f0-c4ba-435f-ba2a-bb7259781b2a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:20.218487 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c894f0-c4ba-435f-ba2a-bb7259781b2a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:20.233716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-97c894f0-c4ba-435f-ba2a-bb7259781b2a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:54:20.233716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:20.233716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:20.233716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:20.233716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:20.233716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:20.233716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:20.233716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:20.233716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:20.233716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:20.233716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:20.233716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:20.233716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:54:20.233716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:54:20.233716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:54:20.233716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:20.234756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:20.236127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:20.236127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:20.236127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:54:20.236127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:54:20.236127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:54:20.236127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:54:20.236127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:54:20.236127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:54:20.236127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:54:20.236127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:20.236127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 546/1091] 10.222.0.83 () {56 vars in 1093 bytes} [Mon Jul 18 14:54:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:20.241611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955815da-37df-456b-af8c-4bf369214232 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:20.308780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:20.343901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955815da-37df-456b-af8c-4bf369214232 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:20.379929 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b67eb-7e08-4877-b7fa-6b43e22c5ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:20.381054 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b67eb-7e08-4877-b7fa-6b43e22c5ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:20.381544 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b67eb-7e08-4877-b7fa-6b43e22c5ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:20.382046 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b67eb-7e08-4877-b7fa-6b43e22c5ca3 None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:20.382952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b67eb-7e08-4877-b7fa-6b43e22c5ca3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:20.383309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b67eb-7e08-4877-b7fa-6b43e22c5ca3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:20.385706 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b67eb-7e08-4877-b7fa-6b43e22c5ca3 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:20.405234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 546/1092] 10.222.0.83 () {56 vars in 1133 bytes} [Mon Jul 18 14:54:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:20.411714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c894f0-c4ba-435f-ba2a-bb7259781b2a None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:20.440525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:20.513729 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c894f0-c4ba-435f-ba2a-bb7259781b2a None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:20.551111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c23e3-b7e2-4653-9b22-91ba441ece7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:20.551737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c23e3-b7e2-4653-9b22-91ba441ece7e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:20.552025 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c23e3-b7e2-4653-9b22-91ba441ece7e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:20.552304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c23e3-b7e2-4653-9b22-91ba441ece7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:20.620322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c23e3-b7e2-4653-9b22-91ba441ece7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:20.621686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c23e3-b7e2-4653-9b22-91ba441ece7e None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:20.649127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:20.721347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 547/1093] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:54:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:54:20.727277 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b67eb-7e08-4877-b7fa-6b43e22c5ca3 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:20.826828 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b67eb-7e08-4877-b7fa-6b43e22c5ca3 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:20.852605 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:20.861190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd06b4-d28d-45ac-a2e4-5a9be89b88a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:20.861986 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd06b4-d28d-45ac-a2e4-5a9be89b88a1 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:20.862274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd06b4-d28d-45ac-a2e4-5a9be89b88a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:20.862684 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd06b4-d28d-45ac-a2e4-5a9be89b88a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:20.873076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd06b4-d28d-45ac-a2e4-5a9be89b88a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:20.874460 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd06b4-d28d-45ac-a2e4-5a9be89b88a1 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:20.893207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-f2fd06b4-d28d-45ac-a2e4-5a9be89b88a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:54:20.893207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:20.893207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:20.893207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:20.893207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:20.893207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:20.893207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:20.893207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:20.893207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:20.893207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:20.893207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:20.893207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:20.893207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:54:20.893207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:54:20.893207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:20.894371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:20.896087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:54:20.896087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:54:20.896087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:54:20.896087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:20.898708 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 547/1094] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:20.900799 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c23e3-b7e2-4653-9b22-91ba441ece7e None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:21.007316 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c23e3-b7e2-4653-9b22-91ba441ece7e None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:21.033179 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:21.041035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec67194-8f0b-475f-9d70-34b1685bf6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:21.041706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec67194-8f0b-475f-9d70-34b1685bf6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:21.041989 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec67194-8f0b-475f-9d70-34b1685bf6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:21.046265 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec67194-8f0b-475f-9d70-34b1685bf6d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:21.046265 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec67194-8f0b-475f-9d70-34b1685bf6d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:21.046265 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec67194-8f0b-475f-9d70-34b1685bf6d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:21.046265 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec67194-8f0b-475f-9d70-34b1685bf6d5 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:21.057849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 548/1095] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:21.063733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd06b4-d28d-45ac-a2e4-5a9be89b88a1 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:21.164257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd06b4-d28d-45ac-a2e4-5a9be89b88a1 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:21.199165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a8d3c-40e5-4b29-bfd0-59aca99a2bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:21.199969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a8d3c-40e5-4b29-bfd0-59aca99a2bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:21.200256 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a8d3c-40e5-4b29-bfd0-59aca99a2bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:21.200535 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a8d3c-40e5-4b29-bfd0-59aca99a2bf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:21.207306 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a8d3c-40e5-4b29-bfd0-59aca99a2bf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:21.208746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a8d3c-40e5-4b29-bfd0-59aca99a2bf3 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:21.222431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-176a8d3c-40e5-4b29-bfd0-59aca99a2bf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:54:21.222431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:21.222431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:21.222431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:21.222431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:21.222431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:21.222431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:21.222431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:21.222431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:21.222431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:21.222431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:21.222431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:21.222431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:54:21.222431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:54:21.222431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:21.223674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:21.224722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:54:21.224722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:54:21.224722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:54:21.224722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:54:21.224722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:54:21.224722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:54:21.224722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:54:21.224722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:21.224722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 548/1096] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:54:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:21.230145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec67194-8f0b-475f-9d70-34b1685bf6d5 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:21.335817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec67194-8f0b-475f-9d70-34b1685bf6d5 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:21.372449 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ef255-e554-4b6c-b5d4-a5c3cbb8abfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:21.373111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ef255-e554-4b6c-b5d4-a5c3cbb8abfa None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:21.373397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ef255-e554-4b6c-b5d4-a5c3cbb8abfa None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:21.373678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ef255-e554-4b6c-b5d4-a5c3cbb8abfa None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:21.374468 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ef255-e554-4b6c-b5d4-a5c3cbb8abfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:21.374813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ef255-e554-4b6c-b5d4-a5c3cbb8abfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:21.376361 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ef255-e554-4b6c-b5d4-a5c3cbb8abfa None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:21.388304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 549/1097] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:54:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:21.395222 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a8d3c-40e5-4b29-bfd0-59aca99a2bf3 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:21.435886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:21.471128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134710ee-92d3-4651-bf37-e62a6097ee7b None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:21.471673 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134710ee-92d3-4651-bf37-e62a6097ee7b None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:21.471673 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134710ee-92d3-4651-bf37-e62a6097ee7b None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:21.473705 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 550/1098] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:54:21.478976 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32017c-6110-4e7f-8b06-112d122ce4c8 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:21.479244 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32017c-6110-4e7f-8b06-112d122ce4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:21.479529 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32017c-6110-4e7f-8b06-112d122ce4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:21.497123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-fd32017c-6110-4e7f-8b06-112d122ce4c8 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:21.498344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-fd32017c-6110-4e7f-8b06-112d122ce4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:21.499107 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a8d3c-40e5-4b29-bfd0-59aca99a2bf3 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:21.533407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b2011-48c9-4d6d-b930-47a89b457be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:21.534062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b2011-48c9-4d6d-b930-47a89b457be5 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:21.534347 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b2011-48c9-4d6d-b930-47a89b457be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:21.534656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b2011-48c9-4d6d-b930-47a89b457be5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:21.536273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b2011-48c9-4d6d-b930-47a89b457be5 None admin] RBAC: Adding query filter params (role.id=f7a8b3159cc4475ebac07df3543883cc, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=7e1355d18fb049468651b3675c217d60) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:21.536754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b2011-48c9-4d6d-b930-47a89b457be5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:21.538848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b2011-48c9-4d6d-b930-47a89b457be5 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:21.547044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:21.547239 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 549/1099] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:54:21 2022] GET /identity/v3/role_assignments?user.id=7e1355d18fb049468651b3675c217d60&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=f7a8b3159cc4475ebac07df3543883cc => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:21.577060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 551/1100] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:54:21.602334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b2011-48c9-4d6d-b930-47a89b457be5 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:21.713519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b2011-48c9-4d6d-b930-47a89b457be5 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:21.715135 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:21.750267 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e6389-2d4a-4781-95ff-3857f9718ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:21.751088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e6389-2d4a-4781-95ff-3857f9718ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:21.751372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e6389-2d4a-4781-95ff-3857f9718ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:21.751648 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e6389-2d4a-4781-95ff-3857f9718ec3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:21.762981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e6389-2d4a-4781-95ff-3857f9718ec3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:54:21.772029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-be0e6389-2d4a-4781-95ff-3857f9718ec3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:54:21.772029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:21.772029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:21.772029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:21.772029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:21.772029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:21.772029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:21.772029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:21.772029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:21.772029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:21.772029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:21.772029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:21.772029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:54:21.772029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:54:21.772029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:54:21.772029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:54:21.773034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:54:21.773034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:54:21.773034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:54:21.773034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:54:21.773034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:21.773034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:21.773034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:21.773034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:21.773034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:21.773034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:21.773034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:21.773034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:21.773034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:21.773034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:21.773034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:21.773034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:21.774150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:21.774150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:21.774150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:21.774150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:21.774150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:54:21.774150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:54:21.774150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:54:21.774150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:54:21.774150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:54:21.774150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:54:21.774150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:54:21.774150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:21.774150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 550/1101] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:54:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:21.779677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32017c-6110-4e7f-8b06-112d122ce4c8 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:21.888023 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32017c-6110-4e7f-8b06-112d122ce4c8 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:21.924332 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33f0d9-7186-46b9-a690-76b62490f538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:21.925019 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33f0d9-7186-46b9-a690-76b62490f538 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:21.925309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33f0d9-7186-46b9-a690-76b62490f538 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:21.925586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33f0d9-7186-46b9-a690-76b62490f538 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:21.926702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33f0d9-7186-46b9-a690-76b62490f538 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:21.927003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33f0d9-7186-46b9-a690-76b62490f538 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:21.928397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33f0d9-7186-46b9-a690-76b62490f538 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:21.939674 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 552/1102] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:54:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:21.945950 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e6389-2d4a-4781-95ff-3857f9718ec3 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:22.062696 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e6389-2d4a-4781-95ff-3857f9718ec3 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:22.099011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e148bc-59af-4973-a534-4d81c3959e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:22.099945 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e148bc-59af-4973-a534-4d81c3959e6e None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:22.100237 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e148bc-59af-4973-a534-4d81c3959e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:22.100507 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e148bc-59af-4973-a534-4d81c3959e6e None admin] PATH_INFO: `/v3/users/dc86af4cb7e64ccab2ad612d419ff507` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:22.101851 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e148bc-59af-4973-a534-4d81c3959e6e None admin] RBAC: Authorizing `identity:get_user(user_id=dc86af4cb7e64ccab2ad612d419ff507)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:22.103392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e148bc-59af-4973-a534-4d81c3959e6e None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:22.105311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 551/1103] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:54:21 2022] GET /identity/v3/users/dc86af4cb7e64ccab2ad612d419ff507 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:22.111455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33f0d9-7186-46b9-a690-76b62490f538 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:22.158594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:22.226798 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33f0d9-7186-46b9-a690-76b62490f538 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:22.263614 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e06ca-e5a9-4870-862d-cb646a0799ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:22.264257 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e06ca-e5a9-4870-862d-cb646a0799ff None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:22.264539 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e06ca-e5a9-4870-862d-cb646a0799ff None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:22.264818 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e06ca-e5a9-4870-862d-cb646a0799ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:22.337762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e06ca-e5a9-4870-862d-cb646a0799ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:22.339248 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e06ca-e5a9-4870-862d-cb646a0799ff None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:22.380906 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:22.458070 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 553/1104] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:54:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:54:22.459509 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e148bc-59af-4973-a534-4d81c3959e6e None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:22.494121 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:22.568922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e148bc-59af-4973-a534-4d81c3959e6e None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:22.610004 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807a056-9e48-42b8-b101-ca911e47b928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:22.610798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807a056-9e48-42b8-b101-ca911e47b928 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:22.611116 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807a056-9e48-42b8-b101-ca911e47b928 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:22.611395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807a056-9e48-42b8-b101-ca911e47b928 None admin] PATH_INFO: `/v3/projects/3f57219ec685451cb2153ec3a41dbdca` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:22.612566 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807a056-9e48-42b8-b101-ca911e47b928 None admin] RBAC: Authorizing `identity:get_project(project_id=3f57219ec685451cb2153ec3a41dbdca)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:22.614094 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807a056-9e48-42b8-b101-ca911e47b928 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:22.615818 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 552/1105] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:54:22 2022] GET /identity/v3/projects/3f57219ec685451cb2153ec3a41dbdca => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:22.622035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e06ca-e5a9-4870-862d-cb646a0799ff None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:22.720659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e06ca-e5a9-4870-862d-cb646a0799ff None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:22.755048 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a7ddf-3832-4252-836c-c275135b20d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:22.755692 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a7ddf-3832-4252-836c-c275135b20d1 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:22.755971 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a7ddf-3832-4252-836c-c275135b20d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:22.756246 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a7ddf-3832-4252-836c-c275135b20d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:22.757123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a7ddf-3832-4252-836c-c275135b20d1 None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=3f57219ec685451cb2153ec3a41dbdca, user.id=dc86af4cb7e64ccab2ad612d419ff507) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:22.757427 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a7ddf-3832-4252-836c-c275135b20d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:22.758916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a7ddf-3832-4252-836c-c275135b20d1 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:22.764737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 554/1106] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:54:22 2022] GET /identity/v3/role_assignments?user.id=dc86af4cb7e64ccab2ad612d419ff507&scope.project.id=3f57219ec685451cb2153ec3a41dbdca&role.id=5891920310b54787bd1152d736d2a3f1 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:24.905106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f06a6-bfed-4c37-b073-bf20f6c04ed8 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:24.905676 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f06a6-bfed-4c37-b073-bf20f6c04ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:24.905676 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f06a6-bfed-4c37-b073-bf20f6c04ed8 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:24.907100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 553/1107] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:24.912496 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba78a59-3ebd-447c-8770-79aebfc427f6 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:24.912966 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba78a59-3ebd-447c-8770-79aebfc427f6 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:24.913136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba78a59-3ebd-447c-8770-79aebfc427f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:24.919151 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:24.931068 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-1ba78a59-3ebd-447c-8770-79aebfc427f6 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:24.932291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1ba78a59-3ebd-447c-8770-79aebfc427f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:24.999004 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 555/1108] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:25.024637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f06a6-bfed-4c37-b073-bf20f6c04ed8 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:25.121023 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:25.125654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f06a6-bfed-4c37-b073-bf20f6c04ed8 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:25.159647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee98529-a69c-4972-967a-45fc5973e405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:25.161500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee98529-a69c-4972-967a-45fc5973e405 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:25.161500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee98529-a69c-4972-967a-45fc5973e405 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:25.162045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee98529-a69c-4972-967a-45fc5973e405 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:25.162998 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee98529-a69c-4972-967a-45fc5973e405 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:25.164434 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee98529-a69c-4972-967a-45fc5973e405 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:25.166062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 554/1109] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:25.172196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba78a59-3ebd-447c-8770-79aebfc427f6 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:25.269145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba78a59-3ebd-447c-8770-79aebfc427f6 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:25.304712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b5610-3853-4d90-a58c-1eeacbeb9af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:25.305367 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b5610-3853-4d90-a58c-1eeacbeb9af4 None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:25.305649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b5610-3853-4d90-a58c-1eeacbeb9af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:25.305923 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b5610-3853-4d90-a58c-1eeacbeb9af4 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:25.306697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b5610-3853-4d90-a58c-1eeacbeb9af4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:25.308180 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b5610-3853-4d90-a58c-1eeacbeb9af4 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:25.314700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:25.398108 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 556/1110] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:54:25 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:54:27.506872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5296b57-210b-4f31-89b3-205eaca16029 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:27.507444 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5296b57-210b-4f31-89b3-205eaca16029 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:27.507444 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5296b57-210b-4f31-89b3-205eaca16029 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:27.508798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 555/1111] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:27.514090 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd6c31-1307-40ce-96ac-ae4649d37c09 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:27.514492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd6c31-1307-40ce-96ac-ae4649d37c09 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:27.514668 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd6c31-1307-40ce-96ac-ae4649d37c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:27.531931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-e8dd6c31-1307-40ce-96ac-ae4649d37c09 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:27.533126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e8dd6c31-1307-40ce-96ac-ae4649d37c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:27.600878 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 557/1112] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:27.624731 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5296b57-210b-4f31-89b3-205eaca16029 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:27.727352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5296b57-210b-4f31-89b3-205eaca16029 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:27.762124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7b70e-72f1-4689-aab9-9470b2ef1943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:27.762929 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7b70e-72f1-4689-aab9-9470b2ef1943 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:27.763218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7b70e-72f1-4689-aab9-9470b2ef1943 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:27.763500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7b70e-72f1-4689-aab9-9470b2ef1943 None admin] PATH_INFO: `/v3/roles/a82e24db1e3c4f868a586e122dae393e` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:27.765010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7b70e-72f1-4689-aab9-9470b2ef1943 None admin] RBAC: Authorizing `identity:get_role(role_id=a82e24db1e3c4f868a586e122dae393e)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:27.766398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7b70e-72f1-4689-aab9-9470b2ef1943 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:27.767818 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 556/1113] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:54:27 2022] GET /identity/v3/roles/a82e24db1e3c4f868a586e122dae393e => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:27.774131 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd6c31-1307-40ce-96ac-ae4649d37c09 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:27.872759 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd6c31-1307-40ce-96ac-ae4649d37c09 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:27.893715 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:27.907713 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c6f98-7cb4-4ffd-bd6c-3495dbb3e569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:27.908683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c6f98-7cb4-4ffd-bd6c-3495dbb3e569 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:27.908970 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c6f98-7cb4-4ffd-bd6c-3495dbb3e569 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:27.909247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c6f98-7cb4-4ffd-bd6c-3495dbb3e569 None admin] PATH_INFO: `/v3/users/f11c88d7bcd44c57806262ba1963f2cf` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:27.922563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c6f98-7cb4-4ffd-bd6c-3495dbb3e569 None admin] RBAC: Authorizing `identity:get_user(user_id=f11c88d7bcd44c57806262ba1963f2cf)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:27.924071 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c6f98-7cb4-4ffd-bd6c-3495dbb3e569 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:27.926945 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 558/1114] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:54:27 2022] GET /identity/v3/users/f11c88d7bcd44c57806262ba1963f2cf => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:27.932389 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7b70e-72f1-4689-aab9-9470b2ef1943 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:27.966654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:28.034249 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7b70e-72f1-4689-aab9-9470b2ef1943 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:28.069200 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dada18-524b-48aa-8a9e-816b9b2eca17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:28.069848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dada18-524b-48aa-8a9e-816b9b2eca17 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:28.070135 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dada18-524b-48aa-8a9e-816b9b2eca17 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:28.070485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dada18-524b-48aa-8a9e-816b9b2eca17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:28.138036 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dada18-524b-48aa-8a9e-816b9b2eca17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:28.139575 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dada18-524b-48aa-8a9e-816b9b2eca17 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:28.171548 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:28.239306 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 557/1115] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:54:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:54:28.245456 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c6f98-7cb4-4ffd-bd6c-3495dbb3e569 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:28.347659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c6f98-7cb4-4ffd-bd6c-3495dbb3e569 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:28.381783 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ef242-c0e0-4a5f-bd08-c0ce6a106ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:28.382679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ef242-c0e0-4a5f-bd08-c0ce6a106ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:28.382971 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ef242-c0e0-4a5f-bd08-c0ce6a106ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:28.383246 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ef242-c0e0-4a5f-bd08-c0ce6a106ff8 None admin] PATH_INFO: `/v3/projects/3f57219ec685451cb2153ec3a41dbdca` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:28.384437 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ef242-c0e0-4a5f-bd08-c0ce6a106ff8 None admin] RBAC: Authorizing `identity:get_project(project_id=3f57219ec685451cb2153ec3a41dbdca)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:28.385969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ef242-c0e0-4a5f-bd08-c0ce6a106ff8 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:28.387724 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 559/1116] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:54:28 2022] GET /identity/v3/projects/3f57219ec685451cb2153ec3a41dbdca => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:28.393987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dada18-524b-48aa-8a9e-816b9b2eca17 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:28.496576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dada18-524b-48aa-8a9e-816b9b2eca17 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:28.530136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567af9b-58ff-4281-b737-97d4bb2cd30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:28.530872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567af9b-58ff-4281-b737-97d4bb2cd30e None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:28.531096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567af9b-58ff-4281-b737-97d4bb2cd30e None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:28.531372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567af9b-58ff-4281-b737-97d4bb2cd30e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:28.532196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567af9b-58ff-4281-b737-97d4bb2cd30e None admin] RBAC: Adding query filter params (role.id=a82e24db1e3c4f868a586e122dae393e, scope.project.id=3f57219ec685451cb2153ec3a41dbdca, user.id=f11c88d7bcd44c57806262ba1963f2cf) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:28.532508 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567af9b-58ff-4281-b737-97d4bb2cd30e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:28.533952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567af9b-58ff-4281-b737-97d4bb2cd30e None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:28.536349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:28.540562 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 558/1117] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:54:28 2022] GET /identity/v3/role_assignments?user.id=f11c88d7bcd44c57806262ba1963f2cf&scope.project.id=3f57219ec685451cb2153ec3a41dbdca&role.id=a82e24db1e3c4f868a586e122dae393e => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:30.588790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3919c8d-0657-4033-a7be-bf47c18051ef None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:30.589355 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3919c8d-0657-4033-a7be-bf47c18051ef None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:30.589355 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3919c8d-0657-4033-a7be-bf47c18051ef None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:30.590743 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 560/1118] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:30.597902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15a530-e98b-487f-bb88-7a804985e1ba None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:30.598364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15a530-e98b-487f-bb88-7a804985e1ba None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:30.598458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15a530-e98b-487f-bb88-7a804985e1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:30.615780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-ea15a530-e98b-487f-bb88-7a804985e1ba None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:30.616983 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ea15a530-e98b-487f-bb88-7a804985e1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:30.683247 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 559/1119] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:30.705788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3919c8d-0657-4033-a7be-bf47c18051ef None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:30.754468 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:30.808370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3919c8d-0657-4033-a7be-bf47c18051ef None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:30.844057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9c2b4-f752-4e26-83e9-7ac725c8bff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:30.844821 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9c2b4-f752-4e26-83e9-7ac725c8bff5 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:30.845100 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9c2b4-f752-4e26-83e9-7ac725c8bff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:30.845388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9c2b4-f752-4e26-83e9-7ac725c8bff5 None admin] PATH_INFO: `/v3/roles/a82e24db1e3c4f868a586e122dae393e` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:30.847612 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9c2b4-f752-4e26-83e9-7ac725c8bff5 None admin] RBAC: Authorizing `identity:get_role(role_id=a82e24db1e3c4f868a586e122dae393e)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:30.849279 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9c2b4-f752-4e26-83e9-7ac725c8bff5 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:30.850730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 561/1120] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:54:30 2022] GET /identity/v3/roles/a82e24db1e3c4f868a586e122dae393e => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:30.856878 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15a530-e98b-487f-bb88-7a804985e1ba None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:30.963236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15a530-e98b-487f-bb88-7a804985e1ba None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:30.990190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:30.997953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14490528-daba-4f4f-8458-d972c05f42e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:30.998918 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14490528-daba-4f4f-8458-d972c05f42e8 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:30.999202 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14490528-daba-4f4f-8458-d972c05f42e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:30.999479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14490528-daba-4f4f-8458-d972c05f42e8 None admin] PATH_INFO: `/v3/users/f11c88d7bcd44c57806262ba1963f2cf` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:31.000669 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14490528-daba-4f4f-8458-d972c05f42e8 None admin] RBAC: Authorizing `identity:get_user(user_id=f11c88d7bcd44c57806262ba1963f2cf)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:31.002115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14490528-daba-4f4f-8458-d972c05f42e8 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:31.004128 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 560/1121] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:54:30 2022] GET /identity/v3/users/f11c88d7bcd44c57806262ba1963f2cf => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:31.010080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9c2b4-f752-4e26-83e9-7ac725c8bff5 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:31.110635 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9c2b4-f752-4e26-83e9-7ac725c8bff5 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:31.124206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:31.145182 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd80b8-954f-4b7b-9f55-3a2f053609ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:31.146035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd80b8-954f-4b7b-9f55-3a2f053609ac None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:31.146318 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd80b8-954f-4b7b-9f55-3a2f053609ac None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:31.146621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd80b8-954f-4b7b-9f55-3a2f053609ac None admin] PATH_INFO: `/v3/projects/3f57219ec685451cb2153ec3a41dbdca` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:31.147843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd80b8-954f-4b7b-9f55-3a2f053609ac None admin] RBAC: Authorizing `identity:get_project(project_id=3f57219ec685451cb2153ec3a41dbdca)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:31.149315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd80b8-954f-4b7b-9f55-3a2f053609ac None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:31.151201 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 562/1122] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:54:31 2022] GET /identity/v3/projects/3f57219ec685451cb2153ec3a41dbdca => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:31.157393 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14490528-daba-4f4f-8458-d972c05f42e8 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:31.262304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14490528-daba-4f4f-8458-d972c05f42e8 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:31.298348 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907291c-4df1-4ec6-9a9c-452418890053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:31.299286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907291c-4df1-4ec6-9a9c-452418890053 None admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:31.299564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907291c-4df1-4ec6-9a9c-452418890053 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:31.299841 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907291c-4df1-4ec6-9a9c-452418890053 None admin] PATH_INFO: `/v3/projects/3f57219ec685451cb2153ec3a41dbdca/users/f11c88d7bcd44c57806262ba1963f2cf/roles/a82e24db1e3c4f868a586e122dae393e` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:31.302225 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907291c-4df1-4ec6-9a9c-452418890053 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f57219ec685451cb2153ec3a41dbdca, user_id=f11c88d7bcd44c57806262ba1963f2cf, role_id=a82e24db1e3c4f868a586e122dae393e)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:31.303699 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907291c-4df1-4ec6-9a9c-452418890053 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:31.376484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 561/1123] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:54:31 2022] PUT /identity/v3/projects/3f57219ec685451cb2153ec3a41dbdca/users/f11c88d7bcd44c57806262ba1963f2cf/roles/a82e24db1e3c4f868a586e122dae393e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:54:33.419583 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604e099-0a6b-43b2-9e6b-122d0ab33513 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:33.420192 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604e099-0a6b-43b2-9e6b-122d0ab33513 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:33.420192 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604e099-0a6b-43b2-9e6b-122d0ab33513 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:33.421593 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 563/1124] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:33.427202 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94fba4-8abb-43e6-8661-d413ecb68a78 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:33.427665 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94fba4-8abb-43e6-8661-d413ecb68a78 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:33.427809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94fba4-8abb-43e6-8661-d413ecb68a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:33.444864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-ee94fba4-8abb-43e6-8661-d413ecb68a78 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:33.446064 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ee94fba4-8abb-43e6-8661-d413ecb68a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:33.495756 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:33.512927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 562/1125] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:33.537236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604e099-0a6b-43b2-9e6b-122d0ab33513 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:33.638455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604e099-0a6b-43b2-9e6b-122d0ab33513 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:33.672580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b8abc-8a77-44d8-8ce6-9536a0fa565e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:33.673364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b8abc-8a77-44d8-8ce6-9536a0fa565e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:33.673649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b8abc-8a77-44d8-8ce6-9536a0fa565e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:33.673940 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b8abc-8a77-44d8-8ce6-9536a0fa565e None admin] PATH_INFO: `/v3/roles/a82e24db1e3c4f868a586e122dae393e` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:33.676069 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b8abc-8a77-44d8-8ce6-9536a0fa565e None admin] RBAC: Authorizing `identity:get_role(role_id=a82e24db1e3c4f868a586e122dae393e)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:33.677476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b8abc-8a77-44d8-8ce6-9536a0fa565e None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:33.678952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 564/1126] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:54:33 2022] GET /identity/v3/roles/a82e24db1e3c4f868a586e122dae393e => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:33.685013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94fba4-8abb-43e6-8661-d413ecb68a78 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:33.782477 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94fba4-8abb-43e6-8661-d413ecb68a78 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:33.816580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f993e-ab2c-4816-b390-37d6051478b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:33.817442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f993e-ab2c-4816-b390-37d6051478b5 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:33.817725 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f993e-ab2c-4816-b390-37d6051478b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:33.818001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f993e-ab2c-4816-b390-37d6051478b5 None admin] PATH_INFO: `/v3/users/f11c88d7bcd44c57806262ba1963f2cf` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:33.819187 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f993e-ab2c-4816-b390-37d6051478b5 None admin] RBAC: Authorizing `identity:get_user(user_id=f11c88d7bcd44c57806262ba1963f2cf)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:33.820625 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f993e-ab2c-4816-b390-37d6051478b5 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:33.822258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 563/1127] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:54:33 2022] GET /identity/v3/users/f11c88d7bcd44c57806262ba1963f2cf => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:33.828204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b8abc-8a77-44d8-8ce6-9536a0fa565e None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:33.878929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:33.931296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b8abc-8a77-44d8-8ce6-9536a0fa565e None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:33.965290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a6827-d0ec-4fe1-adde-c46d1f28654a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:33.965935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a6827-d0ec-4fe1-adde-c46d1f28654a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:33.966220 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a6827-d0ec-4fe1-adde-c46d1f28654a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:33.966498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a6827-d0ec-4fe1-adde-c46d1f28654a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:34.034101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a6827-d0ec-4fe1-adde-c46d1f28654a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:34.035793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a6827-d0ec-4fe1-adde-c46d1f28654a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:34.082848 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:34.136598 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 565/1128] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:54:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:54:34.142710 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f993e-ab2c-4816-b390-37d6051478b5 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:34.199192 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:34.242673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f993e-ab2c-4816-b390-37d6051478b5 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:34.275261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15537be9-f590-4262-b348-f8aeb3141715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:34.276106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15537be9-f590-4262-b348-f8aeb3141715 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:34.276392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15537be9-f590-4262-b348-f8aeb3141715 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:34.276670 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15537be9-f590-4262-b348-f8aeb3141715 None admin] PATH_INFO: `/v3/projects/3f57219ec685451cb2153ec3a41dbdca` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:34.277849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15537be9-f590-4262-b348-f8aeb3141715 None admin] RBAC: Authorizing `identity:get_project(project_id=3f57219ec685451cb2153ec3a41dbdca)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:34.279317 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15537be9-f590-4262-b348-f8aeb3141715 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:34.281048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 564/1129] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:54:34 2022] GET /identity/v3/projects/3f57219ec685451cb2153ec3a41dbdca => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:34.287978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a6827-d0ec-4fe1-adde-c46d1f28654a None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:34.388306 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a6827-d0ec-4fe1-adde-c46d1f28654a None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:34.422360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43058a-5cfa-4994-8a43-3cad985ce069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:34.423044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43058a-5cfa-4994-8a43-3cad985ce069 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:34.423327 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43058a-5cfa-4994-8a43-3cad985ce069 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:34.423605 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43058a-5cfa-4994-8a43-3cad985ce069 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:34.424429 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43058a-5cfa-4994-8a43-3cad985ce069 None admin] RBAC: Adding query filter params (role.id=a82e24db1e3c4f868a586e122dae393e, scope.project.id=3f57219ec685451cb2153ec3a41dbdca, user.id=f11c88d7bcd44c57806262ba1963f2cf) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:34.425576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43058a-5cfa-4994-8a43-3cad985ce069 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:34.427056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43058a-5cfa-4994-8a43-3cad985ce069 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:34.432884 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 566/1130] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:54:34 2022] GET /identity/v3/role_assignments?user.id=f11c88d7bcd44c57806262ba1963f2cf&scope.project.id=3f57219ec685451cb2153ec3a41dbdca&role.id=a82e24db1e3c4f868a586e122dae393e => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:36.521496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5bd82-9b2d-404d-b920-c4d0ec6eb520 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:36.522064 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5bd82-9b2d-404d-b920-c4d0ec6eb520 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:36.522064 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5bd82-9b2d-404d-b920-c4d0ec6eb520 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:36.523547 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 565/1131] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:36.528825 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993a1e0-b86b-4773-9bf2-e625c1506132 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:36.529238 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993a1e0-b86b-4773-9bf2-e625c1506132 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:36.529373 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993a1e0-b86b-4773-9bf2-e625c1506132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:36.547525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-3993a1e0-b86b-4773-9bf2-e625c1506132 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:36.548717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3993a1e0-b86b-4773-9bf2-e625c1506132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:36.558466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:36.623068 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 567/1132] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:36.652200 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5bd82-9b2d-404d-b920-c4d0ec6eb520 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:36.758340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5bd82-9b2d-404d-b920-c4d0ec6eb520 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:36.788404 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:36.799589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e81d8-f030-4799-9135-928df1b77d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:36.800502 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e81d8-f030-4799-9135-928df1b77d4d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:36.800790 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e81d8-f030-4799-9135-928df1b77d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:36.801070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e81d8-f030-4799-9135-928df1b77d4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:36.802265 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e81d8-f030-4799-9135-928df1b77d4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:36.803787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e81d8-f030-4799-9135-928df1b77d4d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:36.805438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 566/1133] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:36.811725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993a1e0-b86b-4773-9bf2-e625c1506132 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:36.913528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993a1e0-b86b-4773-9bf2-e625c1506132 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:36.948878 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b38e16-b157-4174-9d66-13dc39da7b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:36.949545 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b38e16-b157-4174-9d66-13dc39da7b11 None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:36.949831 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b38e16-b157-4174-9d66-13dc39da7b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:36.950110 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b38e16-b157-4174-9d66-13dc39da7b11 None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:36.950849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b38e16-b157-4174-9d66-13dc39da7b11 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:36.952305 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b38e16-b157-4174-9d66-13dc39da7b11 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:36.966507 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:37.120937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 568/1134] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Jul 18 14:54:36 2022] POST /identity/v3/projects => generated 311 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:54:39.214866 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbf33b-7225-44d9-80be-207804938d84 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:39.215416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbf33b-7225-44d9-80be-207804938d84 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:39.215416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbf33b-7225-44d9-80be-207804938d84 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:39.216716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 567/1135] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:39.222501 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfe678-8b3d-402c-a564-3853550b4e74 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:39.222966 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfe678-8b3d-402c-a564-3853550b4e74 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:39.223077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfe678-8b3d-402c-a564-3853550b4e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:39.240730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-84cfe678-8b3d-402c-a564-3853550b4e74 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:39.241931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-84cfe678-8b3d-402c-a564-3853550b4e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:39.310608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 569/1136] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:39.336067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbf33b-7225-44d9-80be-207804938d84 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:39.439956 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbf33b-7225-44d9-80be-207804938d84 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:39.474608 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523195e-5de8-429a-8c12-690935fea7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:39.475356 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523195e-5de8-429a-8c12-690935fea7ee None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:39.475639 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523195e-5de8-429a-8c12-690935fea7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:39.475915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523195e-5de8-429a-8c12-690935fea7ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:39.477088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523195e-5de8-429a-8c12-690935fea7ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:39.478648 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523195e-5de8-429a-8c12-690935fea7ee None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:39.480528 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 568/1137] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:54:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:39.487447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfe678-8b3d-402c-a564-3853550b4e74 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:39.586504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfe678-8b3d-402c-a564-3853550b4e74 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:39.612791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:39.621491 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd98f5-47f5-4604-a49a-044e77d459b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:39.622153 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd98f5-47f5-4604-a49a-044e77d459b0 None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:39.622441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd98f5-47f5-4604-a49a-044e77d459b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:39.622744 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd98f5-47f5-4604-a49a-044e77d459b0 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:39.623454 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd98f5-47f5-4604-a49a-044e77d459b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:39.624898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd98f5-47f5-4604-a49a-044e77d459b0 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:39.700071 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 570/1138] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:54:39 2022] POST /identity/v3/users => generated 286 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:54:41.818686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d25b6b-f0f2-4b04-a0b9-20e73cf94141 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:41.819330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d25b6b-f0f2-4b04-a0b9-20e73cf94141 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:41.819330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d25b6b-f0f2-4b04-a0b9-20e73cf94141 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:41.820622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 569/1139] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:41.826094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef36da-01cc-4d2d-a38c-227c9bffc2d3 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:41.826559 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef36da-01cc-4d2d-a38c-227c9bffc2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:41.826726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef36da-01cc-4d2d-a38c-227c9bffc2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:41.843866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-dcef36da-01cc-4d2d-a38c-227c9bffc2d3 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:41.845185 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-dcef36da-01cc-4d2d-a38c-227c9bffc2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:41.912296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 571/1140] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:41.936684 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d25b6b-f0f2-4b04-a0b9-20e73cf94141 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:41.995944 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:42.037379 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d25b6b-f0f2-4b04-a0b9-20e73cf94141 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:42.071539 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f52fd-4b49-4693-9e5a-f4093a3cd39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:42.072249 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f52fd-4b49-4693-9e5a-f4093a3cd39f None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:42.072539 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f52fd-4b49-4693-9e5a-f4093a3cd39f None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:42.072820 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f52fd-4b49-4693-9e5a-f4093a3cd39f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:42.082989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f52fd-4b49-4693-9e5a-f4093a3cd39f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:54:42.090690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-522f52fd-4b49-4693-9e5a-f4093a3cd39f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:54:42.090690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:42.090690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:42.090690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:42.090690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:42.090690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:42.090690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:42.090690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:42.090690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:42.090690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:42.090690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:42.090690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:42.090690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:54:42.090690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:54:42.090690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:54:42.090690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:54:42.091938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:54:42.091938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:54:42.091938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:54:42.091938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:54:42.091938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:42.091938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:42.091938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:42.091938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:42.091938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:42.091938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:42.091938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:42.091938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:42.091938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:42.091938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:42.091938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:42.091938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:42.093307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:42.093307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:42.093307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:42.093307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:42.093307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:54:42.093307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:54:42.093307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:54:42.093307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:54:42.093307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:54:42.093307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:54:42.093307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:54:42.093307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:42.093307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 570/1141] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:54:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:42.098125 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef36da-01cc-4d2d-a38c-227c9bffc2d3 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:42.197765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef36da-01cc-4d2d-a38c-227c9bffc2d3 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:42.210878 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:42.232088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c7a2f-a3bb-4f16-8ef8-099882347981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:42.233841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7a2f-a3bb-4f16-8ef8-099882347981 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:42.233841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7a2f-a3bb-4f16-8ef8-099882347981 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:42.233841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7a2f-a3bb-4f16-8ef8-099882347981 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:42.234122 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c7a2f-a3bb-4f16-8ef8-099882347981 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:42.234394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c7a2f-a3bb-4f16-8ef8-099882347981 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:42.235823 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c7a2f-a3bb-4f16-8ef8-099882347981 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:42.244605 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 572/1142] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:54:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:42.250597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f52fd-4b49-4693-9e5a-f4093a3cd39f None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:42.351812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f52fd-4b49-4693-9e5a-f4093a3cd39f None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:42.369082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:42.385464 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f00a3-cfbd-4ab6-beb2-4bbdbd09ee1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:42.386335 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f00a3-cfbd-4ab6-beb2-4bbdbd09ee1d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:42.386659 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f00a3-cfbd-4ab6-beb2-4bbdbd09ee1d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:42.386941 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f00a3-cfbd-4ab6-beb2-4bbdbd09ee1d None admin] PATH_INFO: `/v3/users/c3f753ea7eb04a378c5b2fa98b023082` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:42.402646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f00a3-cfbd-4ab6-beb2-4bbdbd09ee1d None admin] RBAC: Authorizing `identity:get_user(user_id=c3f753ea7eb04a378c5b2fa98b023082)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:42.404118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f00a3-cfbd-4ab6-beb2-4bbdbd09ee1d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:42.405846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 571/1143] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:54:42 2022] GET /identity/v3/users/c3f753ea7eb04a378c5b2fa98b023082 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:42.412717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c7a2f-a3bb-4f16-8ef8-099882347981 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:42.513275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c7a2f-a3bb-4f16-8ef8-099882347981 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:42.546665 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40874f-f24e-4da0-bb04-a53e7c2e879e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:42.547300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40874f-f24e-4da0-bb04-a53e7c2e879e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:42.547582 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40874f-f24e-4da0-bb04-a53e7c2e879e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:42.547856 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40874f-f24e-4da0-bb04-a53e7c2e879e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:42.591124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:42.616778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b40874f-f24e-4da0-bb04-a53e7c2e879e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:42.618171 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b40874f-f24e-4da0-bb04-a53e7c2e879e None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:42.733353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 573/1144] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:54:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:54:42.739620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f00a3-cfbd-4ab6-beb2-4bbdbd09ee1d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:42.837986 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f00a3-cfbd-4ab6-beb2-4bbdbd09ee1d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:42.872388 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43661d-423f-405c-8457-b93c65d23f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:42.873176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43661d-423f-405c-8457-b93c65d23f0a None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:42.873465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43661d-423f-405c-8457-b93c65d23f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:42.873744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43661d-423f-405c-8457-b93c65d23f0a None admin] PATH_INFO: `/v3/projects/cbd25fdf71fe408ca0adaa93c33224db` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:42.874968 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e43661d-423f-405c-8457-b93c65d23f0a None admin] RBAC: Authorizing `identity:get_project(project_id=cbd25fdf71fe408ca0adaa93c33224db)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:42.876413 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e43661d-423f-405c-8457-b93c65d23f0a None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:42.878189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 572/1145] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:54:42 2022] GET /identity/v3/projects/cbd25fdf71fe408ca0adaa93c33224db => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:42.884506 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40874f-f24e-4da0-bb04-a53e7c2e879e None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:42.959471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:42.983665 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40874f-f24e-4da0-bb04-a53e7c2e879e None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:43.019573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1698bf-550a-47c2-8c48-ff5e7b477ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:43.020210 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1698bf-550a-47c2-8c48-ff5e7b477ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:43.020492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1698bf-550a-47c2-8c48-ff5e7b477ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:43.020786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1698bf-550a-47c2-8c48-ff5e7b477ac5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:43.021586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1698bf-550a-47c2-8c48-ff5e7b477ac5 None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=cbd25fdf71fe408ca0adaa93c33224db, user.id=c3f753ea7eb04a378c5b2fa98b023082) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:43.021893 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1698bf-550a-47c2-8c48-ff5e7b477ac5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:43.023371 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1698bf-550a-47c2-8c48-ff5e7b477ac5 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:43.028665 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 574/1146] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:54:42 2022] GET /identity/v3/role_assignments?user.id=c3f753ea7eb04a378c5b2fa98b023082&scope.project.id=cbd25fdf71fe408ca0adaa93c33224db&role.id=5891920310b54787bd1152d736d2a3f1 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:45.134887 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e209635-9509-4518-8eee-325cfe088482 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:45.135473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e209635-9509-4518-8eee-325cfe088482 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:45.135473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e209635-9509-4518-8eee-325cfe088482 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:45.136832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 573/1147] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:45.142250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bf3d4-c673-4924-95e8-bd122ac3937c None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:45.143259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bf3d4-c673-4924-95e8-bd122ac3937c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:45.143885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bf3d4-c673-4924-95e8-bd122ac3937c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:45.162378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-446bf3d4-c673-4924-95e8-bd122ac3937c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:45.163694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-446bf3d4-c673-4924-95e8-bd122ac3937c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:45.235570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 575/1148] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:45.258010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e209635-9509-4518-8eee-325cfe088482 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:45.306515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:45.362902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e209635-9509-4518-8eee-325cfe088482 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:45.399252 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d8ad3-2648-41f6-9b27-12048b0feadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:45.400049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d8ad3-2648-41f6-9b27-12048b0feadc None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:45.400383 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d8ad3-2648-41f6-9b27-12048b0feadc None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:45.400676 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d8ad3-2648-41f6-9b27-12048b0feadc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:45.410698 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d8ad3-2648-41f6-9b27-12048b0feadc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:54:45.418402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-e73d8ad3-2648-41f6-9b27-12048b0feadc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:54:45.418402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:45.418402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:45.418402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:45.418402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:45.418402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:45.418402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:45.418402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:45.418402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:45.418402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:45.418402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:45.418402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:45.418402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:54:45.418402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:54:45.418402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:54:45.418402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:54:45.419606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:54:45.419606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:54:45.419606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:54:45.419606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:54:45.419606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:45.419606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:45.419606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:45.419606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:45.419606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:45.419606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:45.419606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:45.419606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:45.419606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:45.419606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:45.419606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:45.419606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:45.425812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:45.425812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:45.425812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:45.425812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:45.425812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:54:45.425812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:54:45.425812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:54:45.425812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:54:45.425812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:54:45.425812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:54:45.425812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:54:45.425812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:45.425812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 574/1149] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:54:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:45.426772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bf3d4-c673-4924-95e8-bd122ac3937c None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:45.493026 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:45.526721 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bf3d4-c673-4924-95e8-bd122ac3937c None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:45.559765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5988b-597e-4133-99c5-8c3585e35656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:45.560430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5988b-597e-4133-99c5-8c3585e35656 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:45.560711 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5988b-597e-4133-99c5-8c3585e35656 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:45.560986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5988b-597e-4133-99c5-8c3585e35656 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:45.561725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5988b-597e-4133-99c5-8c3585e35656 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:45.562030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5988b-597e-4133-99c5-8c3585e35656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:45.563446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5988b-597e-4133-99c5-8c3585e35656 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:45.573203 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 576/1150] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:54:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:45.579419 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d8ad3-2648-41f6-9b27-12048b0feadc None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:45.678802 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d8ad3-2648-41f6-9b27-12048b0feadc None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:45.681129 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:45.712244 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdf145-d712-46b1-91e6-9c60cddfbdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:45.713103 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdf145-d712-46b1-91e6-9c60cddfbdbc None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:45.713386 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdf145-d712-46b1-91e6-9c60cddfbdbc None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:45.713658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdf145-d712-46b1-91e6-9c60cddfbdbc None admin] PATH_INFO: `/v3/users/c3f753ea7eb04a378c5b2fa98b023082` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:45.714870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdf145-d712-46b1-91e6-9c60cddfbdbc None admin] RBAC: Authorizing `identity:get_user(user_id=c3f753ea7eb04a378c5b2fa98b023082)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:45.716356 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdf145-d712-46b1-91e6-9c60cddfbdbc None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:45.717987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 575/1151] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:54:45 2022] GET /identity/v3/users/c3f753ea7eb04a378c5b2fa98b023082 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:45.724113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5988b-597e-4133-99c5-8c3585e35656 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:45.825615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5988b-597e-4133-99c5-8c3585e35656 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:45.854235 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:45.860105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d58b0-1d9c-4664-844f-054d7d5f6334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:45.861073 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d58b0-1d9c-4664-844f-054d7d5f6334 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:45.867892 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d58b0-1d9c-4664-844f-054d7d5f6334 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:45.867892 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d58b0-1d9c-4664-844f-054d7d5f6334 None admin] PATH_INFO: `/v3/projects/cbd25fdf71fe408ca0adaa93c33224db` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:45.867892 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d58b0-1d9c-4664-844f-054d7d5f6334 None admin] RBAC: Authorizing `identity:get_project(project_id=cbd25fdf71fe408ca0adaa93c33224db)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:45.867892 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d58b0-1d9c-4664-844f-054d7d5f6334 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:45.867892 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 577/1152] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:54:45 2022] GET /identity/v3/projects/cbd25fdf71fe408ca0adaa93c33224db => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:45.872307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdf145-d712-46b1-91e6-9c60cddfbdbc None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:45.973907 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdf145-d712-46b1-91e6-9c60cddfbdbc None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:46.008701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0bb59f-34fe-41f1-84aa-7da152434ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:46.009597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bb59f-34fe-41f1-84aa-7da152434ed2 None admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:46.009882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bb59f-34fe-41f1-84aa-7da152434ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:46.010157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bb59f-34fe-41f1-84aa-7da152434ed2 None admin] PATH_INFO: `/v3/projects/cbd25fdf71fe408ca0adaa93c33224db/users/c3f753ea7eb04a378c5b2fa98b023082/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:46.012144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0bb59f-34fe-41f1-84aa-7da152434ed2 None admin] RBAC: Authorizing `identity:create_grant(project_id=cbd25fdf71fe408ca0adaa93c33224db, user_id=c3f753ea7eb04a378c5b2fa98b023082, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:46.013579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0bb59f-34fe-41f1-84aa-7da152434ed2 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:46.097062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 576/1153] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:54:45 2022] PUT /identity/v3/projects/cbd25fdf71fe408ca0adaa93c33224db/users/c3f753ea7eb04a378c5b2fa98b023082/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:54:48.278475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e092a68-7aea-4f38-a8fc-394a176b771c None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:48.279068 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e092a68-7aea-4f38-a8fc-394a176b771c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:48.279068 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e092a68-7aea-4f38-a8fc-394a176b771c None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:48.280544 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 578/1154] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:54:48.286939 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053e0cd-2fed-4fac-be76-4b12c9fc1070 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:48.287500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053e0cd-2fed-4fac-be76-4b12c9fc1070 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:48.287596 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053e0cd-2fed-4fac-be76-4b12c9fc1070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:48.304396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-5053e0cd-2fed-4fac-be76-4b12c9fc1070 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:48.305667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-5053e0cd-2fed-4fac-be76-4b12c9fc1070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:48.332652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:48.374024 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 577/1155] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:48.401978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e092a68-7aea-4f38-a8fc-394a176b771c None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:48.504020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e092a68-7aea-4f38-a8fc-394a176b771c None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:48.537545 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764f445-272d-496c-9ead-2a7976034083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:48.538251 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764f445-272d-496c-9ead-2a7976034083 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:48.538548 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764f445-272d-496c-9ead-2a7976034083 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:48.538840 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764f445-272d-496c-9ead-2a7976034083 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:48.549194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764f445-272d-496c-9ead-2a7976034083 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:54:48.556842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-9764f445-272d-496c-9ead-2a7976034083 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:54:48.556842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:48.556842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:48.556842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:48.556842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:48.556842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:48.556842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:48.556842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:48.556842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:48.556842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:48.556842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:48.556842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:48.556842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:54:48.556842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:54:48.556842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:54:48.556842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:54:48.558105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:54:48.558105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:54:48.558105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:54:48.558105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:54:48.558105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:48.558105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:48.558105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:48.558105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:48.558105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:48.558105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:48.558105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:48.558105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:48.558105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:48.558105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:48.558105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:48.558105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:48.559157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:48.559157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:48.559157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:48.559157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:48.559157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:54:48.559157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:54:48.559157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:54:48.559157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:54:48.559157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:54:48.559157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:54:48.559157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:54:48.559157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:54:48.559157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 579/1156] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:54:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:48.564995 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053e0cd-2fed-4fac-be76-4b12c9fc1070 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:48.665234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053e0cd-2fed-4fac-be76-4b12c9fc1070 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:48.699859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8182af1-3450-4c40-81eb-915818732d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:48.700513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8182af1-3450-4c40-81eb-915818732d0d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:48.700795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8182af1-3450-4c40-81eb-915818732d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:48.701072 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8182af1-3450-4c40-81eb-915818732d0d None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:48.701810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8182af1-3450-4c40-81eb-915818732d0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:48.702115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8182af1-3450-4c40-81eb-915818732d0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:48.703523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8182af1-3450-4c40-81eb-915818732d0d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:48.711517 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 578/1157] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:54:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:48.717494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764f445-272d-496c-9ead-2a7976034083 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:48.777583 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:48.821052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764f445-272d-496c-9ead-2a7976034083 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:48.854157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03366444-9cdd-44d3-b870-52d5bc5969cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:48.855054 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03366444-9cdd-44d3-b870-52d5bc5969cd None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:48.855341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03366444-9cdd-44d3-b870-52d5bc5969cd None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:48.855621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03366444-9cdd-44d3-b870-52d5bc5969cd None admin] PATH_INFO: `/v3/users/c3f753ea7eb04a378c5b2fa98b023082` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:48.856813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03366444-9cdd-44d3-b870-52d5bc5969cd None admin] RBAC: Authorizing `identity:get_user(user_id=c3f753ea7eb04a378c5b2fa98b023082)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:48.858286 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03366444-9cdd-44d3-b870-52d5bc5969cd None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:48.860023 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 580/1158] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:54:48 2022] GET /identity/v3/users/c3f753ea7eb04a378c5b2fa98b023082 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:48.867382 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8182af1-3450-4c40-81eb-915818732d0d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:48.893105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:48.969074 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8182af1-3450-4c40-81eb-915818732d0d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:49.004039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d5f06-0a25-449a-9dfc-48469b06f709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:49.004686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d5f06-0a25-449a-9dfc-48469b06f709 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:49.004969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d5f06-0a25-449a-9dfc-48469b06f709 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:49.005245 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d5f06-0a25-449a-9dfc-48469b06f709 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:49.071613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d5f06-0a25-449a-9dfc-48469b06f709 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:49.072985 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d5f06-0a25-449a-9dfc-48469b06f709 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:49.096563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:49.170642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 579/1159] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:54:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:54:49.177802 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03366444-9cdd-44d3-b870-52d5bc5969cd None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:49.278424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03366444-9cdd-44d3-b870-52d5bc5969cd None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:49.305261 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:49.313708 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba09dd3-a1ae-4a78-9bd1-1dba941ed7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:49.314866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba09dd3-a1ae-4a78-9bd1-1dba941ed7ee None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:49.317556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba09dd3-a1ae-4a78-9bd1-1dba941ed7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:49.317556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba09dd3-a1ae-4a78-9bd1-1dba941ed7ee None admin] PATH_INFO: `/v3/projects/cbd25fdf71fe408ca0adaa93c33224db` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:49.317556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba09dd3-a1ae-4a78-9bd1-1dba941ed7ee None admin] RBAC: Authorizing `identity:get_project(project_id=cbd25fdf71fe408ca0adaa93c33224db)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:49.318066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba09dd3-a1ae-4a78-9bd1-1dba941ed7ee None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:49.319812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 581/1160] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:54:49 2022] GET /identity/v3/projects/cbd25fdf71fe408ca0adaa93c33224db => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:54:49.326332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d5f06-0a25-449a-9dfc-48469b06f709 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:49.425435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d5f06-0a25-449a-9dfc-48469b06f709 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:49.447827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:49.458668 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d57b8-b06f-429e-bf34-77cc08dffe0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:49.459325 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d57b8-b06f-429e-bf34-77cc08dffe0a None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:49.459608 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d57b8-b06f-429e-bf34-77cc08dffe0a None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:49.459889 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d57b8-b06f-429e-bf34-77cc08dffe0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:49.460719 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d57b8-b06f-429e-bf34-77cc08dffe0a None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=cbd25fdf71fe408ca0adaa93c33224db, user.id=c3f753ea7eb04a378c5b2fa98b023082) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:49.461033 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d57b8-b06f-429e-bf34-77cc08dffe0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:49.462471 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d57b8-b06f-429e-bf34-77cc08dffe0a None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:49.468318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 580/1161] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:54:49 2022] GET /identity/v3/role_assignments?user.id=c3f753ea7eb04a378c5b2fa98b023082&scope.project.id=cbd25fdf71fe408ca0adaa93c33224db&role.id=5891920310b54787bd1152d736d2a3f1 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:51.583220 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf1f49-d91a-4a11-abf3-efe67a4364c6 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:51.583789 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf1f49-d91a-4a11-abf3-efe67a4364c6 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:51.583789 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf1f49-d91a-4a11-abf3-efe67a4364c6 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:51.585169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 582/1162] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:51.590473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78e8c8-e064-45e8-b910-84867e39a2ed None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:51.590987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78e8c8-e064-45e8-b910-84867e39a2ed None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:51.591093 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78e8c8-e064-45e8-b910-84867e39a2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:51.607811 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-0c78e8c8-e064-45e8-b910-84867e39a2ed None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:51.609024 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0c78e8c8-e064-45e8-b910-84867e39a2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:51.675641 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 581/1163] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:51.698874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf1f49-d91a-4a11-abf3-efe67a4364c6 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:51.816926 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf1f49-d91a-4a11-abf3-efe67a4364c6 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:51.853961 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5534fa-6b0d-420b-83b2-ca1dfaac78c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:51.854620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5534fa-6b0d-420b-83b2-ca1dfaac78c9 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:51.854963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5534fa-6b0d-420b-83b2-ca1dfaac78c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:51.855296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5534fa-6b0d-420b-83b2-ca1dfaac78c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:51.922986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:51.931613 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5534fa-6b0d-420b-83b2-ca1dfaac78c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:51.933278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5534fa-6b0d-420b-83b2-ca1dfaac78c9 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:52.045581 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 583/1164] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:54:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:54:52.051872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78e8c8-e064-45e8-b910-84867e39a2ed None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:52.153996 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78e8c8-e064-45e8-b910-84867e39a2ed None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:52.170648 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:52.186775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9705dc-27a3-4ed5-8711-5c38e46aa746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:52.187580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9705dc-27a3-4ed5-8711-5c38e46aa746 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:52.187870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9705dc-27a3-4ed5-8711-5c38e46aa746 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:52.188150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9705dc-27a3-4ed5-8711-5c38e46aa746 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:52.195304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9705dc-27a3-4ed5-8711-5c38e46aa746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:52.196754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9705dc-27a3-4ed5-8711-5c38e46aa746 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-8e9705dc-27a3-4ed5-8711-5c38e46aa746 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:54:52.212910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:54:52.214290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:54:52.214290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:54:52.214290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:54:52.214290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:54:52.214290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:54:52.214290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:54:52.214290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:54:52.214290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:54:52.214290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:54:52.214290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:54:52.214290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:54:52.214290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:54:52.214290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:54:52.214290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:54:52.214290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:54:52.214290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:54:52.215255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:54:52.215255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:54:52.215255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:54:52.215255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:54:52.215255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:54:52.215255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:54:52.215255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:54:52.215255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:54:52.215255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:54:52.215255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:54:52.215255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 18 14:54:52.215255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:54:52.215255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 14:54:52.215255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:54:52.215255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 582/1165] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Jul 18 14:54:52 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:54:52.220665 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5534fa-6b0d-420b-83b2-ca1dfaac78c9 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:52.317904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:52.324684 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5534fa-6b0d-420b-83b2-ca1dfaac78c9 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:52.360136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6c1a9-13a7-4625-bd0b-8311e5c7c8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:52.361417 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6c1a9-13a7-4625-bd0b-8311e5c7c8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:52.361697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6c1a9-13a7-4625-bd0b-8311e5c7c8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:52.361973 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6c1a9-13a7-4625-bd0b-8311e5c7c8e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:52.362725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6c1a9-13a7-4625-bd0b-8311e5c7c8e4 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:54:52.363031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6c1a9-13a7-4625-bd0b-8311e5c7c8e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:52.364487 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6c1a9-13a7-4625-bd0b-8311e5c7c8e4 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:52.371844 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 584/1166] 10.222.0.83 () {56 vars in 1115 bytes} [Mon Jul 18 14:54:52 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:54.573008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15598ea-36c6-4993-aa94-d3d2311da6a4 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:54.573605 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15598ea-36c6-4993-aa94-d3d2311da6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:54.573605 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15598ea-36c6-4993-aa94-d3d2311da6a4 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:54.574963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 583/1167] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:54.580229 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b0841-b2d7-4a2b-ae5b-656f81e814ed None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:54.580670 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b0841-b2d7-4a2b-ae5b-656f81e814ed None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:54.580789 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b0841-b2d7-4a2b-ae5b-656f81e814ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:54.598183 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-243b0841-b2d7-4a2b-ae5b-656f81e814ed None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:54.599419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-243b0841-b2d7-4a2b-ae5b-656f81e814ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:54.665590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 585/1168] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:54:54.692093 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15598ea-36c6-4993-aa94-d3d2311da6a4 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:54.795868 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15598ea-36c6-4993-aa94-d3d2311da6a4 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:54.830061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b7a1f-1dd6-4448-b204-c9c3f355458b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:54.830676 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b7a1f-1dd6-4448-b204-c9c3f355458b None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:54.830956 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b7a1f-1dd6-4448-b204-c9c3f355458b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:54.831231 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b7a1f-1dd6-4448-b204-c9c3f355458b None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:54.831997 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b7a1f-1dd6-4448-b204-c9c3f355458b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:54.833372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b7a1f-1dd6-4448-b204-c9c3f355458b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:54.911706 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 584/1169] 10.222.0.83 () {60 vars in 1138 bytes} [Mon Jul 18 14:54:54 2022] POST /identity/v3/domains => generated 267 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:54:56.998123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d170025-0b79-460e-9c0d-187f18a41d70 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:56.998813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d170025-0b79-460e-9c0d-187f18a41d70 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:56.998813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d170025-0b79-460e-9c0d-187f18a41d70 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:57.000137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 586/1170] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:57.005573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23c059-0950-444b-8267-4109148aa0a6 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:57.006007 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23c059-0950-444b-8267-4109148aa0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:57.006140 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23c059-0950-444b-8267-4109148aa0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:57.023262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-3a23c059-0950-444b-8267-4109148aa0a6 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:57.024454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-3a23c059-0950-444b-8267-4109148aa0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:57.054447 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:57.093933 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 585/1171] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:54:57.119574 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d170025-0b79-460e-9c0d-187f18a41d70 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:57.211899 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:57.224671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d170025-0b79-460e-9c0d-187f18a41d70 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:57.263093 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f77faf-cb7c-4771-a8f2-c02be4a91eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:57.263955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f77faf-cb7c-4771-a8f2-c02be4a91eec None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:57.264234 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f77faf-cb7c-4771-a8f2-c02be4a91eec None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:57.264510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f77faf-cb7c-4771-a8f2-c02be4a91eec None admin] PATH_INFO: `/v3/domains/d3462f225c0b43bea783e9c8fc63b75c` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:57.275519 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f77faf-cb7c-4771-a8f2-c02be4a91eec None admin] RBAC: Authorizing `identity:get_domain(domain_id=d3462f225c0b43bea783e9c8fc63b75c)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:57.277045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f77faf-cb7c-4771-a8f2-c02be4a91eec None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:57.278828 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 587/1172] 10.222.0.83 () {56 vars in 1183 bytes} [Mon Jul 18 14:54:57 2022] GET /identity/v3/domains/d3462f225c0b43bea783e9c8fc63b75c => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:57.285601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23c059-0950-444b-8267-4109148aa0a6 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:57.387840 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23c059-0950-444b-8267-4109148aa0a6 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:57.421233 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc4130-93c1-4a73-ba51-7cdde235dc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:57.421882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc4130-93c1-4a73-ba51-7cdde235dc26 None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:57.422165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc4130-93c1-4a73-ba51-7cdde235dc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:57.422454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc4130-93c1-4a73-ba51-7cdde235dc26 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:57.423841 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc4130-93c1-4a73-ba51-7cdde235dc26 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:57.425329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc4130-93c1-4a73-ba51-7cdde235dc26 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:57.522557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 586/1173] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Jul 18 14:54:57 2022] POST /identity/v3/projects => generated 361 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:54:59.622745 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf96ebc-fb25-4735-a6a4-815d92ab43ef None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:59.623309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf96ebc-fb25-4735-a6a4-815d92ab43ef None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:59.623309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf96ebc-fb25-4735-a6a4-815d92ab43ef None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:59.624711 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 588/1174] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:54:59.630005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6c33-0f9c-47c2-a3e0-fcffd60921a9 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:59.630455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6c33-0f9c-47c2-a3e0-fcffd60921a9 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:59.630617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6c33-0f9c-47c2-a3e0-fcffd60921a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:59.647618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b9fa6c33-0f9c-47c2-a3e0-fcffd60921a9 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:54:59.648826 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b9fa6c33-0f9c-47c2-a3e0-fcffd60921a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:54:59.663159 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:54:59.715524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 587/1175] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:54:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:54:59.741845 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf96ebc-fb25-4735-a6a4-815d92ab43ef None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:59.842473 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf96ebc-fb25-4735-a6a4-815d92ab43ef None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:54:59.876934 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6c163-9b5b-4b93-b3e8-86ae0b5528a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:54:59.877718 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6c163-9b5b-4b93-b3e8-86ae0b5528a9 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:54:59.877995 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6c163-9b5b-4b93-b3e8-86ae0b5528a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:54:59.878268 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6c163-9b5b-4b93-b3e8-86ae0b5528a9 None admin] PATH_INFO: `/v3/domains/d3462f225c0b43bea783e9c8fc63b75c` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:54:59.879470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6c163-9b5b-4b93-b3e8-86ae0b5528a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d3462f225c0b43bea783e9c8fc63b75c)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:54:59.881053 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6c163-9b5b-4b93-b3e8-86ae0b5528a9 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:54:59.882654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 589/1176] 10.222.0.83 () {56 vars in 1183 bytes} [Mon Jul 18 14:54:59 2022] GET /identity/v3/domains/d3462f225c0b43bea783e9c8fc63b75c => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:54:59.888552 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa6c33-0f9c-47c2-a3e0-fcffd60921a9 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:54:59.987701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa6c33-0f9c-47c2-a3e0-fcffd60921a9 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:00.022139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dfe1e-f49a-42ff-b795-f6b2149f01ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:00.022914 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dfe1e-f49a-42ff-b795-f6b2149f01ab None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:00.023201 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dfe1e-f49a-42ff-b795-f6b2149f01ab None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:00.023482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dfe1e-f49a-42ff-b795-f6b2149f01ab None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:00.024205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373dfe1e-f49a-42ff-b795-f6b2149f01ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:55:00.025643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373dfe1e-f49a-42ff-b795-f6b2149f01ab None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:55:00.153009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 588/1177] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:54:59 2022] POST /identity/v3/users => generated 311 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:55:02.316882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07aa4b-66bc-4625-8fb9-27ee7614972c None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:02.317479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07aa4b-66bc-4625-8fb9-27ee7614972c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:02.317479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07aa4b-66bc-4625-8fb9-27ee7614972c None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:02.318888 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 590/1178] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:55:02.324064 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b571b-89d4-4262-a15d-6edc2e0bd9c9 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:02.324482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b571b-89d4-4262-a15d-6edc2e0bd9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:02.324633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b571b-89d4-4262-a15d-6edc2e0bd9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:02.336269 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:55:02.341602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-110b571b-89d4-4262-a15d-6edc2e0bd9c9 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:55:02.342862 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-110b571b-89d4-4262-a15d-6edc2e0bd9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:55:02.412772 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 589/1179] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:55:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:55:02.437556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07aa4b-66bc-4625-8fb9-27ee7614972c None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:02.457706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:55:02.549076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07aa4b-66bc-4625-8fb9-27ee7614972c None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:02.585356 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52dbc8c-e8ee-49cd-8667-55af581f64b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:02.586110 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dbc8c-e8ee-49cd-8667-55af581f64b8 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:02.586395 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dbc8c-e8ee-49cd-8667-55af581f64b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:02.586700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dbc8c-e8ee-49cd-8667-55af581f64b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:02.597299 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52dbc8c-e8ee-49cd-8667-55af581f64b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:55:02.605205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-f52dbc8c-e8ee-49cd-8667-55af581f64b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:55:02.605205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:55:02.605205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:55:02.605205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:55:02.605205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:55:02.605205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:55:02.605205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:55:02.605205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:55:02.605205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:55:02.605205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:55:02.605205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:55:02.605205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:55:02.605205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:55:02.605205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:55:02.605205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:55:02.605205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:55:02.606479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:55:02.606479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:55:02.606479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:55:02.606479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:55:02.606479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:55:02.606479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:55:02.606479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:55:02.606479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:55:02.606479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:55:02.606479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:55:02.606479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:55:02.606479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:55:02.606479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:55:02.606479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:55:02.606479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:55:02.606479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:55:02.607556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:55:02.607556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:55:02.607556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:55:02.607556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:55:02.607556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:55:02.607556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:55:02.607556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:55:02.607556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:55:02.607556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:55:02.607556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:55:02.607556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:55:02.607556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:55:02.607556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 591/1180] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:55:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:55:02.613470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b571b-89d4-4262-a15d-6edc2e0bd9c9 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:02.709748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b571b-89d4-4262-a15d-6edc2e0bd9c9 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:02.743853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba340d14-16d9-42df-a1db-4a0c0332ebaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:02.744487 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba340d14-16d9-42df-a1db-4a0c0332ebaf None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:02.744769 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba340d14-16d9-42df-a1db-4a0c0332ebaf None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:02.745045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba340d14-16d9-42df-a1db-4a0c0332ebaf None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:02.745784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba340d14-16d9-42df-a1db-4a0c0332ebaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:55:02.746088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba340d14-16d9-42df-a1db-4a0c0332ebaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:55:02.747485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba340d14-16d9-42df-a1db-4a0c0332ebaf None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:55:02.749624 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:55:02.755974 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 590/1181] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:55:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:55:02.762012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52dbc8c-e8ee-49cd-8667-55af581f64b8 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:02.839806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:55:02.864230 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52dbc8c-e8ee-49cd-8667-55af581f64b8 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:02.897106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96f0e7-efd4-4c93-95b0-71766eefb184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:02.897972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96f0e7-efd4-4c93-95b0-71766eefb184 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:02.898258 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96f0e7-efd4-4c93-95b0-71766eefb184 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:02.898667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96f0e7-efd4-4c93-95b0-71766eefb184 None admin] PATH_INFO: `/v3/users/6603865238a24873b60b3fb27494f114` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:02.912312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96f0e7-efd4-4c93-95b0-71766eefb184 None admin] RBAC: Authorizing `identity:get_user(user_id=6603865238a24873b60b3fb27494f114)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:55:02.913798 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96f0e7-efd4-4c93-95b0-71766eefb184 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:55:02.915509 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 592/1182] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:55:02 2022] GET /identity/v3/users/6603865238a24873b60b3fb27494f114 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:55:02.921505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba340d14-16d9-42df-a1db-4a0c0332ebaf None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:03.025858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba340d14-16d9-42df-a1db-4a0c0332ebaf None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:03.059927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608b512-23aa-4cae-b70a-8888991488e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:03.060562 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608b512-23aa-4cae-b70a-8888991488e7 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:03.060846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608b512-23aa-4cae-b70a-8888991488e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:03.061124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608b512-23aa-4cae-b70a-8888991488e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:03.124574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:55:03.128658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608b512-23aa-4cae-b70a-8888991488e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:55:03.130108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608b512-23aa-4cae-b70a-8888991488e7 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:55:03.226646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 591/1183] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:55:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:55:03.232978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96f0e7-efd4-4c93-95b0-71766eefb184 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:03.335447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96f0e7-efd4-4c93-95b0-71766eefb184 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:03.370085 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685c41b-d091-4cfa-9a69-7f3652b5a54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:03.370973 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685c41b-d091-4cfa-9a69-7f3652b5a54a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:03.371264 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685c41b-d091-4cfa-9a69-7f3652b5a54a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:03.371545 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685c41b-d091-4cfa-9a69-7f3652b5a54a None admin] PATH_INFO: `/v3/projects/dbd06f2904b3492f928eb19326ee36d8` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:03.372734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685c41b-d091-4cfa-9a69-7f3652b5a54a None admin] RBAC: Authorizing `identity:get_project(project_id=dbd06f2904b3492f928eb19326ee36d8)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:55:03.374192 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685c41b-d091-4cfa-9a69-7f3652b5a54a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:55:03.376145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 593/1184] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:55:03 2022] GET /identity/v3/projects/dbd06f2904b3492f928eb19326ee36d8 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:55:03.382371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608b512-23aa-4cae-b70a-8888991488e7 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:03.481186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:55:03.485386 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608b512-23aa-4cae-b70a-8888991488e7 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:03.518955 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95af019-e01e-4803-957b-2c79a41860b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:03.519594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95af019-e01e-4803-957b-2c79a41860b8 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:03.519882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95af019-e01e-4803-957b-2c79a41860b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:03.520161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95af019-e01e-4803-957b-2c79a41860b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:03.520984 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95af019-e01e-4803-957b-2c79a41860b8 None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=dbd06f2904b3492f928eb19326ee36d8, user.id=6603865238a24873b60b3fb27494f114) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:55:03.521343 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95af019-e01e-4803-957b-2c79a41860b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:55:03.522815 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95af019-e01e-4803-957b-2c79a41860b8 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:55:03.528414 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 592/1185] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:55:03 2022] GET /identity/v3/role_assignments?user.id=6603865238a24873b60b3fb27494f114&scope.project.id=dbd06f2904b3492f928eb19326ee36d8&role.id=5891920310b54787bd1152d736d2a3f1 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:55:05.540363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a58c58-820d-403f-bd23-575068cb485e None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:05.540363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a58c58-820d-403f-bd23-575068cb485e None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:05.540363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a58c58-820d-403f-bd23-575068cb485e None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:05.540363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 594/1186] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:55:05.544913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d47c5-8b72-4c59-987f-8ff1ba047185 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:05.545407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d47c5-8b72-4c59-987f-8ff1ba047185 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:05.545503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d47c5-8b72-4c59-987f-8ff1ba047185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:05.563322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-a21d47c5-8b72-4c59-987f-8ff1ba047185 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:55:05.564514 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a21d47c5-8b72-4c59-987f-8ff1ba047185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:55:05.630882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 593/1187] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:55:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:55:05.654272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a58c58-820d-403f-bd23-575068cb485e None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:05.657542 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:55:05.758612 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a58c58-820d-403f-bd23-575068cb485e None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:05.792470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045c689-7f0a-4e28-b001-f872e32eea89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:05.793187 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045c689-7f0a-4e28-b001-f872e32eea89 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:05.793471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045c689-7f0a-4e28-b001-f872e32eea89 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:05.793748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045c689-7f0a-4e28-b001-f872e32eea89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:05.804298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045c689-7f0a-4e28-b001-f872e32eea89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:55:05.812097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-9045c689-7f0a-4e28-b001-f872e32eea89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:55:05.812097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:55:05.812097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:55:05.812097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:55:05.812097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:55:05.812097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:55:05.812097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:55:05.812097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:55:05.812097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:55:05.812097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:55:05.812097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:55:05.812097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:55:05.812097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:55:05.812097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:55:05.812097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:55:05.812097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:55:05.813843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:55:05.813843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:55:05.813843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:55:05.813843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:55:05.813843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:55:05.813843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:55:05.813843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:55:05.813843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:55:05.813843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:55:05.813843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:55:05.813843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:55:05.813843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:55:05.813843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:55:05.813843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:55:05.813843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:55:05.813843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:55:05.814806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:55:05.814806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:55:05.814806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:55:05.814806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:55:05.814806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:55:05.814806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:55:05.814806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:55:05.814806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:55:05.814806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:55:05.814806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:55:05.814806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:55:05.814806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:55:05.814806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 595/1188] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:55:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:55:05.820396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d47c5-8b72-4c59-987f-8ff1ba047185 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:05.911343 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:55:05.920334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d47c5-8b72-4c59-987f-8ff1ba047185 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:05.956268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab88cb-7bc6-40fb-b180-f6131c3c4a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:05.956941 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab88cb-7bc6-40fb-b180-f6131c3c4a1d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:05.957232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab88cb-7bc6-40fb-b180-f6131c3c4a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:05.957515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab88cb-7bc6-40fb-b180-f6131c3c4a1d None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:05.958263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab88cb-7bc6-40fb-b180-f6131c3c4a1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:55:05.958611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab88cb-7bc6-40fb-b180-f6131c3c4a1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:55:05.959996 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab88cb-7bc6-40fb-b180-f6131c3c4a1d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:55:05.968389 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 594/1189] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:55:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:55:05.974618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045c689-7f0a-4e28-b001-f872e32eea89 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:06.038203 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:55:06.077427 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045c689-7f0a-4e28-b001-f872e32eea89 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:06.110308 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c49115-d430-49be-9a74-e7fadb6a3bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:06.111209 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c49115-d430-49be-9a74-e7fadb6a3bae None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:06.111490 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c49115-d430-49be-9a74-e7fadb6a3bae None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:06.111763 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c49115-d430-49be-9a74-e7fadb6a3bae None admin] PATH_INFO: `/v3/users/6603865238a24873b60b3fb27494f114` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:06.112982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c49115-d430-49be-9a74-e7fadb6a3bae None admin] RBAC: Authorizing `identity:get_user(user_id=6603865238a24873b60b3fb27494f114)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:55:06.114461 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c49115-d430-49be-9a74-e7fadb6a3bae None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:55:06.116142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 596/1190] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:55:05 2022] GET /identity/v3/users/6603865238a24873b60b3fb27494f114 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:55:06.122210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab88cb-7bc6-40fb-b180-f6131c3c4a1d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:06.231855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab88cb-7bc6-40fb-b180-f6131c3c4a1d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:06.267083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4097c26-45ae-4db7-86e0-a7bb95554ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:06.267928 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4097c26-45ae-4db7-86e0-a7bb95554ceb None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:06.268213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4097c26-45ae-4db7-86e0-a7bb95554ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:06.268487 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4097c26-45ae-4db7-86e0-a7bb95554ceb None admin] PATH_INFO: `/v3/projects/dbd06f2904b3492f928eb19326ee36d8` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:06.269652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4097c26-45ae-4db7-86e0-a7bb95554ceb None admin] RBAC: Authorizing `identity:get_project(project_id=dbd06f2904b3492f928eb19326ee36d8)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:55:06.271120 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4097c26-45ae-4db7-86e0-a7bb95554ceb None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:55:06.272820 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 595/1191] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:55:06 2022] GET /identity/v3/projects/dbd06f2904b3492f928eb19326ee36d8 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:55:06.279232 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c49115-d430-49be-9a74-e7fadb6a3bae None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:06.380068 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c49115-d430-49be-9a74-e7fadb6a3bae None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:06.408557 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:55:06.414075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1952392-e31b-4473-b800-251ebf12c88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:06.415011 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1952392-e31b-4473-b800-251ebf12c88f None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:06.415290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1952392-e31b-4473-b800-251ebf12c88f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:06.415565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1952392-e31b-4473-b800-251ebf12c88f None admin] PATH_INFO: `/v3/projects/dbd06f2904b3492f928eb19326ee36d8/users/6603865238a24873b60b3fb27494f114/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:06.417490 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1952392-e31b-4473-b800-251ebf12c88f None admin] RBAC: Authorizing `identity:create_grant(project_id=dbd06f2904b3492f928eb19326ee36d8, user_id=6603865238a24873b60b3fb27494f114, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:55:06.419039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1952392-e31b-4473-b800-251ebf12c88f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:55:06.511633 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 597/1192] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:55:06 2022] PUT /identity/v3/projects/dbd06f2904b3492f928eb19326ee36d8/users/6603865238a24873b60b3fb27494f114/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:55:08.572791 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4633b1a-5991-4f88-887b-b209f7abf20c None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:08.573427 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4633b1a-5991-4f88-887b-b209f7abf20c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:08.573427 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4633b1a-5991-4f88-887b-b209f7abf20c None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:08.574770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 596/1193] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:55:08.580730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478022c-b60c-47ec-93ed-1f343b2c1a7f None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:08.581179 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478022c-b60c-47ec-93ed-1f343b2c1a7f None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:08.581290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478022c-b60c-47ec-93ed-1f343b2c1a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:08.598618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-3478022c-b60c-47ec-93ed-1f343b2c1a7f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:55:08.599827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3478022c-b60c-47ec-93ed-1f343b2c1a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:55:08.666736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 598/1194] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:55:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:55:08.690925 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4633b1a-5991-4f88-887b-b209f7abf20c None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:08.716605 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:55:08.794077 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4633b1a-5991-4f88-887b-b209f7abf20c None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:08.829177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe8797-8002-48c3-b480-a7c90ac9b713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:08.829905 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe8797-8002-48c3-b480-a7c90ac9b713 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:08.830191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe8797-8002-48c3-b480-a7c90ac9b713 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:08.830471 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe8797-8002-48c3-b480-a7c90ac9b713 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:08.840442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe8797-8002-48c3-b480-a7c90ac9b713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:55:08.848652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-3efe8797-8002-48c3-b480-a7c90ac9b713 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:55:08.848652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:55:08.848652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:55:08.848652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:55:08.848652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:55:08.848652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:55:08.848652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:55:08.848652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:55:08.848652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:55:08.848652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:55:08.848652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:55:08.848652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:55:08.848652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:55:08.848652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:55:08.848652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:55:08.848652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:55:08.849953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:55:08.849953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:55:08.849953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:55:08.849953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:55:08.849953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:55:08.849953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:55:08.849953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:55:08.849953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:55:08.849953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:55:08.849953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:55:08.849953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:55:08.849953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:55:08.849953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:55:08.849953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:55:08.849953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:55:08.849953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:55:08.851506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:55:08.851506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:55:08.851506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:55:08.851506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:55:08.851506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:55:08.851506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:55:08.851506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:55:08.851506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:55:08.851506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:55:08.851506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:55:08.851506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:55:08.851506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:55:08.851506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 597/1195] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:55:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:55:08.856676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478022c-b60c-47ec-93ed-1f343b2c1a7f None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:08.955286 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478022c-b60c-47ec-93ed-1f343b2c1a7f None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:08.976015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:55:08.990660 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11739ccd-226a-4b9a-bc02-eaf9576651f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:08.991333 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11739ccd-226a-4b9a-bc02-eaf9576651f7 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:08.991620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11739ccd-226a-4b9a-bc02-eaf9576651f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:08.991896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11739ccd-226a-4b9a-bc02-eaf9576651f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:08.992639 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11739ccd-226a-4b9a-bc02-eaf9576651f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:55:08.992943 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11739ccd-226a-4b9a-bc02-eaf9576651f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:55:08.994415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11739ccd-226a-4b9a-bc02-eaf9576651f7 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:55:09.003225 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 599/1196] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:55:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:55:09.009135 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe8797-8002-48c3-b480-a7c90ac9b713 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:09.092804 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:55:09.111376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe8797-8002-48c3-b480-a7c90ac9b713 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:09.145580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e30ca-a7b7-4660-9457-1ed8f73acaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:09.146447 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e30ca-a7b7-4660-9457-1ed8f73acaa8 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:09.146763 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e30ca-a7b7-4660-9457-1ed8f73acaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:09.147042 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e30ca-a7b7-4660-9457-1ed8f73acaa8 None admin] PATH_INFO: `/v3/users/6603865238a24873b60b3fb27494f114` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:09.148281 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e30ca-a7b7-4660-9457-1ed8f73acaa8 None admin] RBAC: Authorizing `identity:get_user(user_id=6603865238a24873b60b3fb27494f114)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:55:09.149789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e30ca-a7b7-4660-9457-1ed8f73acaa8 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:55:09.151481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 598/1197] 10.222.0.83 () {56 vars in 1177 bytes} [Mon Jul 18 14:55:09 2022] GET /identity/v3/users/6603865238a24873b60b3fb27494f114 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:55:09.157603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11739ccd-226a-4b9a-bc02-eaf9576651f7 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:09.259339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11739ccd-226a-4b9a-bc02-eaf9576651f7 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:09.298411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725e8cb-2a16-43c6-a44e-d04cab989b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:09.299083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725e8cb-2a16-43c6-a44e-d04cab989b31 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:09.299365 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725e8cb-2a16-43c6-a44e-d04cab989b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:09.299640 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725e8cb-2a16-43c6-a44e-d04cab989b31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:09.349959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:55:09.368733 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725e8cb-2a16-43c6-a44e-d04cab989b31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:55:09.370198 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725e8cb-2a16-43c6-a44e-d04cab989b31 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:55:09.467439 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 600/1198] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:55:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:55:09.474298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e30ca-a7b7-4660-9457-1ed8f73acaa8 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:09.573581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e30ca-a7b7-4660-9457-1ed8f73acaa8 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:09.607382 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1f518-d530-40b5-a7f2-487a823088b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:09.608163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1f518-d530-40b5-a7f2-487a823088b2 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:09.608449 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1f518-d530-40b5-a7f2-487a823088b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:09.608728 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1f518-d530-40b5-a7f2-487a823088b2 None admin] PATH_INFO: `/v3/projects/dbd06f2904b3492f928eb19326ee36d8` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:09.610610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1f518-d530-40b5-a7f2-487a823088b2 None admin] RBAC: Authorizing `identity:get_project(project_id=dbd06f2904b3492f928eb19326ee36d8)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:55:09.612050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1f518-d530-40b5-a7f2-487a823088b2 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:55:09.613868 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 599/1199] 10.222.0.83 () {56 vars in 1186 bytes} [Mon Jul 18 14:55:09 2022] GET /identity/v3/projects/dbd06f2904b3492f928eb19326ee36d8 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:55:09.620110 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725e8cb-2a16-43c6-a44e-d04cab989b31 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:55:09.712899 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:55:09.731729 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725e8cb-2a16-43c6-a44e-d04cab989b31 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:55:09.766803 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b1b14-95c9-4a7d-89c9-617f4a3e4d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:55:09.767473 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b1b14-95c9-4a7d-89c9-617f4a3e4d0c None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:55:09.767754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b1b14-95c9-4a7d-89c9-617f4a3e4d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:55:09.768029 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b1b14-95c9-4a7d-89c9-617f4a3e4d0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:55:09.768850 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b1b14-95c9-4a7d-89c9-617f4a3e4d0c None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=dbd06f2904b3492f928eb19326ee36d8, user.id=6603865238a24873b60b3fb27494f114) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:55:09.769158 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b1b14-95c9-4a7d-89c9-617f4a3e4d0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:55:09.770622 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b1b14-95c9-4a7d-89c9-617f4a3e4d0c None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:55:09.776419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 601/1200] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:55:09 2022] GET /identity/v3/role_assignments?user.id=6603865238a24873b60b3fb27494f114&scope.project.id=dbd06f2904b3492f928eb19326ee36d8&role.id=5891920310b54787bd1152d736d2a3f1 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:55:19.603100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=91 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:55:19.773795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=27 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:31.819420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b10ce-b243-40c2-8344-4782ba3b30a8 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:31.820423 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b10ce-b243-40c2-8344-4782ba3b30a8 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:31.820423 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b10ce-b243-40c2-8344-4782ba3b30a8 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:31.822186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 600/1201] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:56:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:56:31.827799 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1164aa6-ccc1-4739-a9c1-000c95635cfd None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:31.828288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1164aa6-ccc1-4739-a9c1-000c95635cfd None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:31.828403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1164aa6-ccc1-4739-a9c1-000c95635cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:31.846468 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-d1164aa6-ccc1-4739-a9c1-000c95635cfd None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:56:31.848645 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d1164aa6-ccc1-4739-a9c1-000c95635cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:56:31.916683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 602/1202] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:56:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:56:31.941956 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b10ce-b243-40c2-8344-4782ba3b30a8 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:32.049236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b10ce-b243-40c2-8344-4782ba3b30a8 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:32.084274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e71d9-4f65-4f68-86fc-5a94933e990b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:32.084990 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e71d9-4f65-4f68-86fc-5a94933e990b None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:32.085279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e71d9-4f65-4f68-86fc-5a94933e990b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:32.085563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e71d9-4f65-4f68-86fc-5a94933e990b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:32.095526 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e71d9-4f65-4f68-86fc-5a94933e990b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:32.097280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e71d9-4f65-4f68-86fc-5a94933e990b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:32.115361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-7c2e71d9-4f65-4f68-86fc-5a94933e990b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:32.115361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:32.115361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:32.115361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:32.115361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:32.115361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:32.115361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:32.115361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:32.115361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:32.115361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:32.115361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:32.115361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:32.115361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:56:32.115361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:56:32.115361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:32.116570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:32.117775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:56:32.117775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:56:32.117775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:32.117775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:56:32.117775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 601/1203] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:56:32.123053 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1164aa6-ccc1-4739-a9c1-000c95635cfd None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:32.224986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1164aa6-ccc1-4739-a9c1-000c95635cfd None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:32.253379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:32.259607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b52da3-ea5b-4208-ad5a-d5a59a9f64e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:32.262123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b52da3-ea5b-4208-ad5a-d5a59a9f64e9 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:32.262123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b52da3-ea5b-4208-ad5a-d5a59a9f64e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:32.262123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b52da3-ea5b-4208-ad5a-d5a59a9f64e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:32.262123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b52da3-ea5b-4208-ad5a-d5a59a9f64e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:32.262123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b52da3-ea5b-4208-ad5a-d5a59a9f64e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:32.263531 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b52da3-ea5b-4208-ad5a-d5a59a9f64e9 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:32.275776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 603/1204] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:32.281725 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e71d9-4f65-4f68-86fc-5a94933e990b None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:32.335901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:32.381646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e71d9-4f65-4f68-86fc-5a94933e990b None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:32.414220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ef69f-2742-429c-b70c-100c60028b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:32.414915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ef69f-2742-429c-b70c-100c60028b6a None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:32.415202 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ef69f-2742-429c-b70c-100c60028b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:32.415483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ef69f-2742-429c-b70c-100c60028b6a None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:32.416200 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ef69f-2742-429c-b70c-100c60028b6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:32.417640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ef69f-2742-429c-b70c-100c60028b6a None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:32.574268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 602/1205] 10.222.0.83 () {60 vars in 1132 bytes} [Mon Jul 18 14:56:32 2022] POST /identity/v3/users => generated 251 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:56:34.667627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea607e67-0339-4e2e-bb9e-aa1d64b48a91 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:34.668219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea607e67-0339-4e2e-bb9e-aa1d64b48a91 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:34.668219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea607e67-0339-4e2e-bb9e-aa1d64b48a91 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:34.669607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 604/1206] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:56:34.674810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff76550-d8e2-4b8f-8231-0bb849f03535 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:34.675343 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff76550-d8e2-4b8f-8231-0bb849f03535 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:34.675450 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff76550-d8e2-4b8f-8231-0bb849f03535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:34.692958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-aff76550-d8e2-4b8f-8231-0bb849f03535 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:56:34.694177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-aff76550-d8e2-4b8f-8231-0bb849f03535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:56:34.762771 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 603/1207] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:56:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:56:34.787757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea607e67-0339-4e2e-bb9e-aa1d64b48a91 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:34.901841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea607e67-0339-4e2e-bb9e-aa1d64b48a91 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:34.936682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892ed46-645f-4bfb-afc4-6a5f50f64277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:34.937493 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892ed46-645f-4bfb-afc4-6a5f50f64277 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:34.937776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892ed46-645f-4bfb-afc4-6a5f50f64277 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:34.938073 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892ed46-645f-4bfb-afc4-6a5f50f64277 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:34.949170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892ed46-645f-4bfb-afc4-6a5f50f64277 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:56:34.957807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-9892ed46-645f-4bfb-afc4-6a5f50f64277 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:56:34.957807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:34.957807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:34.957807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:34.957807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:34.957807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:34.957807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:34.957807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:34.957807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:34.957807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:34.957807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:34.957807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:34.957807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:56:34.957807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:56:34.957807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:56:34.957807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:56:34.959067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:56:34.959067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:56:34.959067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:56:34.959067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:56:34.959067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:34.959067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:34.959067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:34.959067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:34.959067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:34.959067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:34.959067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:34.959067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:34.959067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:34.959067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:34.959067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:34.959067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:34.959978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:34.959978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:34.959978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:34.959978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:34.959978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:56:34.959978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:56:34.959978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:56:34.959978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:56:34.959978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:56:34.959978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:56:34.959978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:56:34.959978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:56:34.959978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 605/1208] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:56:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:34.965471 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff76550-d8e2-4b8f-8231-0bb849f03535 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:35.007298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:35.067384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff76550-d8e2-4b8f-8231-0bb849f03535 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:35.102148 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7201c05-743f-4e7d-9672-3708a28c1ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:35.102851 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7201c05-743f-4e7d-9672-3708a28c1ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:35.103139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7201c05-743f-4e7d-9672-3708a28c1ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:35.103420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7201c05-743f-4e7d-9672-3708a28c1ef2 None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:35.104171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7201c05-743f-4e7d-9672-3708a28c1ef2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:35.104482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7201c05-743f-4e7d-9672-3708a28c1ef2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:35.105865 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7201c05-743f-4e7d-9672-3708a28c1ef2 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:35.114922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 604/1209] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:56:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:35.120798 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892ed46-645f-4bfb-afc4-6a5f50f64277 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:35.178284 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:35.232423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892ed46-645f-4bfb-afc4-6a5f50f64277 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:35.266181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05dbce9-cda5-4407-8489-79aebfa89a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:35.267024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05dbce9-cda5-4407-8489-79aebfa89a57 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:35.267310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05dbce9-cda5-4407-8489-79aebfa89a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:35.267587 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05dbce9-cda5-4407-8489-79aebfa89a57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:35.281675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05dbce9-cda5-4407-8489-79aebfa89a57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:35.283284 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05dbce9-cda5-4407-8489-79aebfa89a57 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:35.301604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-d05dbce9-cda5-4407-8489-79aebfa89a57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:35.301604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:35.301604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:35.301604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:35.301604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:35.301604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:35.301604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:35.301604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:35.301604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:35.301604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:35.301604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:35.301604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:35.301604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:56:35.301604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:56:35.301604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:35.302751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:35.303765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:56:35.303765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:56:35.303765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:35.303765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:56:35.303765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 606/1210] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:35.308655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7201c05-743f-4e7d-9672-3708a28c1ef2 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:35.407533 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:35.411263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7201c05-743f-4e7d-9672-3708a28c1ef2 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:35.445548 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf39b5-9225-4b77-921a-575915934cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:35.446179 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf39b5-9225-4b77-921a-575915934cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:35.446473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf39b5-9225-4b77-921a-575915934cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:35.446873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf39b5-9225-4b77-921a-575915934cf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:35.447599 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf39b5-9225-4b77-921a-575915934cf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:35.447908 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf39b5-9225-4b77-921a-575915934cf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:35.449324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf39b5-9225-4b77-921a-575915934cf1 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:35.460865 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 605/1211] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:35.466873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05dbce9-cda5-4407-8489-79aebfa89a57 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:35.580922 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05dbce9-cda5-4407-8489-79aebfa89a57 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:35.582873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:35.617767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f81492-61b1-490d-8599-1175c52af918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:35.618586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f81492-61b1-490d-8599-1175c52af918 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:35.618873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f81492-61b1-490d-8599-1175c52af918 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:35.619146 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f81492-61b1-490d-8599-1175c52af918 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:35.626142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f81492-61b1-490d-8599-1175c52af918 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:35.628108 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f81492-61b1-490d-8599-1175c52af918 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:35.645394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-19f81492-61b1-490d-8599-1175c52af918 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:56:35.645394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:35.645394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:35.645394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:35.645394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:35.645394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:35.645394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:35.645394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:35.645394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:35.645394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:35.645394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:35.645394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:35.645394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:56:35.645394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:56:35.645394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:56:35.645394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:35.652664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:35.653904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:35.653904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:35.653904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:56:35.653904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:56:35.653904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:56:35.653904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:56:35.653904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:56:35.653904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:56:35.653904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:56:35.653904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:56:35.653904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 607/1212] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:35.654721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf39b5-9225-4b77-921a-575915934cf1 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:35.757927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf39b5-9225-4b77-921a-575915934cf1 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:35.793363 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e55027-f0f7-496c-8f04-e048b2503142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:35.794016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e55027-f0f7-496c-8f04-e048b2503142 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:35.794297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e55027-f0f7-496c-8f04-e048b2503142 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:35.794593 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e55027-f0f7-496c-8f04-e048b2503142 None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:35.795561 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e55027-f0f7-496c-8f04-e048b2503142 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:35.795874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e55027-f0f7-496c-8f04-e048b2503142 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:35.797311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e55027-f0f7-496c-8f04-e048b2503142 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:35.816488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 606/1213] 10.222.0.83 () {56 vars in 1143 bytes} [Mon Jul 18 14:56:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:35.822488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f81492-61b1-490d-8599-1175c52af918 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:35.923306 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f81492-61b1-490d-8599-1175c52af918 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:35.959127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae299b-d82d-450e-8e63-4bf8c9f3caf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:35.959770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae299b-d82d-450e-8e63-4bf8c9f3caf3 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:35.960051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae299b-d82d-450e-8e63-4bf8c9f3caf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:35.960326 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae299b-d82d-450e-8e63-4bf8c9f3caf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:35.991230 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:36.030380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae299b-d82d-450e-8e63-4bf8c9f3caf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:36.031811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae299b-d82d-450e-8e63-4bf8c9f3caf3 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:36.134761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 608/1214] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:56:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:56:36.140755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e55027-f0f7-496c-8f04-e048b2503142 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:36.162093 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:36.242261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e55027-f0f7-496c-8f04-e048b2503142 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:36.277531 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e3b68-cfde-4300-afd8-5f46185eeb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:36.278330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e3b68-cfde-4300-afd8-5f46185eeb60 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:36.278649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e3b68-cfde-4300-afd8-5f46185eeb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:36.278933 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e3b68-cfde-4300-afd8-5f46185eeb60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:36.289877 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e3b68-cfde-4300-afd8-5f46185eeb60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:36.291317 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e3b68-cfde-4300-afd8-5f46185eeb60 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:36.308654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-761e3b68-cfde-4300-afd8-5f46185eeb60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:36.308654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:36.308654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:36.308654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:36.308654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:36.308654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:36.308654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:36.308654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:36.308654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:36.308654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:36.308654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:36.308654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:36.308654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:56:36.308654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:56:36.308654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:36.309829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:36.316037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:56:36.316037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:56:36.316037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:36.316037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:56:36.316037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 607/1215] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:36.316397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae299b-d82d-450e-8e63-4bf8c9f3caf3 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:36.378455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:36.420361 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae299b-d82d-450e-8e63-4bf8c9f3caf3 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:36.453996 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870ec60-008d-4146-b8e2-e9a12e67557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:36.454642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870ec60-008d-4146-b8e2-e9a12e67557a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:36.454928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870ec60-008d-4146-b8e2-e9a12e67557a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:36.455204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870ec60-008d-4146-b8e2-e9a12e67557a None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:36.456596 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870ec60-008d-4146-b8e2-e9a12e67557a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:36.456895 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870ec60-008d-4146-b8e2-e9a12e67557a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:36.458329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870ec60-008d-4146-b8e2-e9a12e67557a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:36.470171 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 609/1216] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:36.477141 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e3b68-cfde-4300-afd8-5f46185eeb60 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:36.548057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:36.579115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e3b68-cfde-4300-afd8-5f46185eeb60 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:36.614819 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c790b-458e-4655-8f20-77fe28fbb2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:36.615794 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c790b-458e-4655-8f20-77fe28fbb2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:36.615924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c790b-458e-4655-8f20-77fe28fbb2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:36.616199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c790b-458e-4655-8f20-77fe28fbb2a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:36.623057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c790b-458e-4655-8f20-77fe28fbb2a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:36.624527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c790b-458e-4655-8f20-77fe28fbb2a1 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:36.639191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-801c790b-458e-4655-8f20-77fe28fbb2a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:56:36.639191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:36.639191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:36.639191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:36.639191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:36.639191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:36.639191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:36.639191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:36.639191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:36.639191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:36.639191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:36.639191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:36.639191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:56:36.639191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:56:36.639191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:36.644784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:36.645935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:56:36.645935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:56:36.645935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:56:36.645935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:56:36.645935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:56:36.645935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:56:36.645935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:56:36.645935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:56:36.645935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 608/1217] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:56:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:36.646657 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870ec60-008d-4146-b8e2-e9a12e67557a None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:36.750409 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870ec60-008d-4146-b8e2-e9a12e67557a None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:36.774089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:36.785520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05b04c-9c2d-4a23-b94e-d27e7d576b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:36.786197 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05b04c-9c2d-4a23-b94e-d27e7d576b2d None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:36.786479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05b04c-9c2d-4a23-b94e-d27e7d576b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:36.786786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05b04c-9c2d-4a23-b94e-d27e7d576b2d None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:36.787613 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05b04c-9c2d-4a23-b94e-d27e7d576b2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:36.787917 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05b04c-9c2d-4a23-b94e-d27e7d576b2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:36.789361 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05b04c-9c2d-4a23-b94e-d27e7d576b2d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:36.801339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 610/1218] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:56:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:36.807710 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c790b-458e-4655-8f20-77fe28fbb2a1 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:36.910393 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c790b-458e-4655-8f20-77fe28fbb2a1 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:36.941840 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:36.947634 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa281fd1-1da4-4133-9c66-b77cd394a0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:36.948285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa281fd1-1da4-4133-9c66-b77cd394a0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:36.948571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa281fd1-1da4-4133-9c66-b77cd394a0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:36.948849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa281fd1-1da4-4133-9c66-b77cd394a0b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:36.949667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa281fd1-1da4-4133-9c66-b77cd394a0b1 None admin] RBAC: Adding query filter params (role.id=29b8b950372848ffa9a75e778ee66b4c, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=76cc0ae48b5a49c6a7a1c11a08c389e4) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:36.949975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa281fd1-1da4-4133-9c66-b77cd394a0b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:36.951462 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa281fd1-1da4-4133-9c66-b77cd394a0b1 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:36.957325 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 609/1219] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:56:36 2022] GET /identity/v3/role_assignments?user.id=76cc0ae48b5a49c6a7a1c11a08c389e4&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=29b8b950372848ffa9a75e778ee66b4c => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:39.047460 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a10de-fa12-4c56-a337-97755ef42172 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:39.048017 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a10de-fa12-4c56-a337-97755ef42172 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:39.048017 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a10de-fa12-4c56-a337-97755ef42172 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:39.050062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 611/1220] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:56:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:56:39.055682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d12520-b509-461e-8c01-c23d111753fc None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:39.056131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d12520-b509-461e-8c01-c23d111753fc None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:39.056230 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d12520-b509-461e-8c01-c23d111753fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:39.074255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b4d12520-b509-461e-8c01-c23d111753fc None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:56:39.075504 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b4d12520-b509-461e-8c01-c23d111753fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:56:39.141361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 610/1221] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:56:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:56:39.165040 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a10de-fa12-4c56-a337-97755ef42172 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:39.273406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a10de-fa12-4c56-a337-97755ef42172 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:39.308317 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db376556-7a89-430b-bb21-4f4b96dbf86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:39.309110 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db376556-7a89-430b-bb21-4f4b96dbf86b None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:39.318057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db376556-7a89-430b-bb21-4f4b96dbf86b None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:39.318057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db376556-7a89-430b-bb21-4f4b96dbf86b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:39.320492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db376556-7a89-430b-bb21-4f4b96dbf86b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:56:39.328405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-db376556-7a89-430b-bb21-4f4b96dbf86b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:56:39.328405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:39.328405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:39.328405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:39.328405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:39.328405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:39.328405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:39.328405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:39.328405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:39.328405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:39.328405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:39.328405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:39.328405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:56:39.328405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:56:39.328405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:56:39.328405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:56:39.329642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:56:39.329642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:56:39.329642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:56:39.329642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:56:39.329642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:39.329642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:39.329642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:39.329642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:39.329642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:39.329642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:39.329642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:39.329642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:39.329642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:39.329642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:39.329642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:39.329642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:39.330896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:39.330896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:39.330896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:39.330896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:39.330896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:56:39.330896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:56:39.330896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:56:39.330896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:56:39.330896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:56:39.330896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:56:39.330896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:56:39.330896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:56:39.330896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 612/1222] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:56:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:39.336383 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d12520-b509-461e-8c01-c23d111753fc None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:39.439326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d12520-b509-461e-8c01-c23d111753fc None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:39.464597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:39.476328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c11d9-ee2a-46c3-9078-10ddc20929cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:39.477026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c11d9-ee2a-46c3-9078-10ddc20929cc None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:39.477311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c11d9-ee2a-46c3-9078-10ddc20929cc None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:39.477590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c11d9-ee2a-46c3-9078-10ddc20929cc None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:39.478341 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c11d9-ee2a-46c3-9078-10ddc20929cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:39.478696 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c11d9-ee2a-46c3-9078-10ddc20929cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:39.499894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c11d9-ee2a-46c3-9078-10ddc20929cc None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:39.499894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 611/1223] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:56:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:56:39.500100 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db376556-7a89-430b-bb21-4f4b96dbf86b None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:39.538782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:39.595353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db376556-7a89-430b-bb21-4f4b96dbf86b None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:39.629578 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c261dfa-f573-448b-9bf3-72a66ca2082e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:39.630408 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c261dfa-f573-448b-9bf3-72a66ca2082e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:39.630723 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c261dfa-f573-448b-9bf3-72a66ca2082e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:39.631002 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c261dfa-f573-448b-9bf3-72a66ca2082e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:39.642572 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c261dfa-f573-448b-9bf3-72a66ca2082e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:39.644146 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c261dfa-f573-448b-9bf3-72a66ca2082e None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:39.661430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-4c261dfa-f573-448b-9bf3-72a66ca2082e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:39.661430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:39.661430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:39.661430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:39.661430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:39.661430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:39.661430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:39.661430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:39.661430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:39.661430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:39.661430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:39.661430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:39.661430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:56:39.661430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:56:39.661430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:39.662647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:39.663805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:56:39.663805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:56:39.663805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:39.663805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:56:39.663805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 613/1224] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:39.668704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c11d9-ee2a-46c3-9078-10ddc20929cc None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:39.772106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c11d9-ee2a-46c3-9078-10ddc20929cc None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:39.805967 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cc6c0-6c19-40e6-a5dc-05c1c0c06a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:39.806568 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cc6c0-6c19-40e6-a5dc-05c1c0c06a7c None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:39.806860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cc6c0-6c19-40e6-a5dc-05c1c0c06a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:39.807133 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cc6c0-6c19-40e6-a5dc-05c1c0c06a7c None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:39.807847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cc6c0-6c19-40e6-a5dc-05c1c0c06a7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:39.808152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cc6c0-6c19-40e6-a5dc-05c1c0c06a7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:39.809522 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cc6c0-6c19-40e6-a5dc-05c1c0c06a7c None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:39.821714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 612/1225] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:39.827626 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c261dfa-f573-448b-9bf3-72a66ca2082e None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:39.929075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:39.935928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c261dfa-f573-448b-9bf3-72a66ca2082e None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:39.970655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d6537-efb5-4925-ad7c-3c93c72855ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:39.972063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d6537-efb5-4925-ad7c-3c93c72855ff None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:39.972345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d6537-efb5-4925-ad7c-3c93c72855ff None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:39.972886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d6537-efb5-4925-ad7c-3c93c72855ff None admin] PATH_INFO: `/v3/users/placement` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:39.980092 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d6537-efb5-4925-ad7c-3c93c72855ff None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:39.981592 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d6537-efb5-4925-ad7c-3c93c72855ff None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:39.997530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-4d6d6537-efb5-4925-ad7c-3c93c72855ff None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:56:39.997530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:39.997530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:39.997530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:39.997530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:39.997530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:39.997530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:39.997530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:39.997530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:39.997530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:39.997530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:39.997530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:39.997530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:56:39.997530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:56:39.997530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:56:39.997530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:39.998936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:40.000348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:40.000348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:40.000348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:56:40.000348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:56:40.000348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:56:40.000348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:56:40.000348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:56:40.000348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:56:40.000348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:56:40.000348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:56:40.002712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 614/1226] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:40.004859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cc6c0-6c19-40e6-a5dc-05c1c0c06a7c None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:40.055491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:40.107189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cc6c0-6c19-40e6-a5dc-05c1c0c06a7c None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:40.152058 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25699a12-c3b7-41a0-b079-7f085ca3585a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:40.152729 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25699a12-c3b7-41a0-b079-7f085ca3585a None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:40.153008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25699a12-c3b7-41a0-b079-7f085ca3585a None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:40.153279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25699a12-c3b7-41a0-b079-7f085ca3585a None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:40.154111 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25699a12-c3b7-41a0-b079-7f085ca3585a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:40.154436 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25699a12-c3b7-41a0-b079-7f085ca3585a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:40.156136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25699a12-c3b7-41a0-b079-7f085ca3585a None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:40.173678 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 613/1227] 10.222.0.83 () {56 vars in 1143 bytes} [Mon Jul 18 14:56:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:40.180298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d6537-efb5-4925-ad7c-3c93c72855ff None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:40.285766 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d6537-efb5-4925-ad7c-3c93c72855ff None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:40.321437 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6453878-c3a1-4e59-8b71-3b0940c17be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:40.323088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6453878-c3a1-4e59-8b71-3b0940c17be6 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:40.323088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6453878-c3a1-4e59-8b71-3b0940c17be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:40.323088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6453878-c3a1-4e59-8b71-3b0940c17be6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:40.334333 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6453878-c3a1-4e59-8b71-3b0940c17be6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:40.335841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6453878-c3a1-4e59-8b71-3b0940c17be6 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:40.354849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-b6453878-c3a1-4e59-8b71-3b0940c17be6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:40.354849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:40.354849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:40.354849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:40.354849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:40.354849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:40.354849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:40.354849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:40.354849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:40.354849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:40.354849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:40.354849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:40.354849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:56:40.354849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:56:40.354849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:40.355969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:40.357083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:56:40.357083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:56:40.357083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:40.357083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:56:40.357083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 615/1228] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:40.364045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25699a12-c3b7-41a0-b079-7f085ca3585a None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:40.470485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25699a12-c3b7-41a0-b079-7f085ca3585a None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:40.473990 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:40.505566 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96640920-c6fb-465a-9ce0-b165f4fd8d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:40.506226 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96640920-c6fb-465a-9ce0-b165f4fd8d0a None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:40.506517 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96640920-c6fb-465a-9ce0-b165f4fd8d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:40.506828 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96640920-c6fb-465a-9ce0-b165f4fd8d0a None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:40.507549 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96640920-c6fb-465a-9ce0-b165f4fd8d0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:40.507856 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96640920-c6fb-465a-9ce0-b165f4fd8d0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:40.509240 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96640920-c6fb-465a-9ce0-b165f4fd8d0a None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:40.521769 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 614/1229] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:40.528130 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6453878-c3a1-4e59-8b71-3b0940c17be6 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:40.542431 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:40.639593 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6453878-c3a1-4e59-8b71-3b0940c17be6 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:40.677136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fba1df-4956-4b0f-9985-c6eefb92212c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:40.678056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fba1df-4956-4b0f-9985-c6eefb92212c None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:40.678345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fba1df-4956-4b0f-9985-c6eefb92212c None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:40.678651 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fba1df-4956-4b0f-9985-c6eefb92212c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:40.685626 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fba1df-4956-4b0f-9985-c6eefb92212c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:40.687118 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fba1df-4956-4b0f-9985-c6eefb92212c None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:40.701322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-a5fba1df-4956-4b0f-9985-c6eefb92212c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:56:40.701322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:40.701322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:40.701322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:40.701322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:40.701322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:40.701322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:40.701322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:40.701322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:40.701322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:40.701322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:40.701322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:40.701322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:56:40.701322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:56:40.701322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:40.702416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:40.703590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:56:40.703590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:56:40.703590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:56:40.703590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:56:40.703590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:56:40.703590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:56:40.703590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:56:40.703590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:56:40.706713 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 616/1230] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:56:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:40.709256 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96640920-c6fb-465a-9ce0-b165f4fd8d0a None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:40.812260 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96640920-c6fb-465a-9ce0-b165f4fd8d0a None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:40.847204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97063635-aa9d-4200-ae2d-f55be17fd78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:40.848130 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97063635-aa9d-4200-ae2d-f55be17fd78a None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:40.848418 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97063635-aa9d-4200-ae2d-f55be17fd78a None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:40.848700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97063635-aa9d-4200-ae2d-f55be17fd78a None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:40.849488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97063635-aa9d-4200-ae2d-f55be17fd78a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:40.849801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97063635-aa9d-4200-ae2d-f55be17fd78a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:40.851278 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97063635-aa9d-4200-ae2d-f55be17fd78a None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:40.864029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 615/1231] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:56:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:40.870553 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fba1df-4956-4b0f-9985-c6eefb92212c None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:40.938080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:40.974751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fba1df-4956-4b0f-9985-c6eefb92212c None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:41.009539 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cdfd8-8138-4d26-8f42-0a109143bd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:41.010436 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cdfd8-8138-4d26-8f42-0a109143bd5d None admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:41.010754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cdfd8-8138-4d26-8f42-0a109143bd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:41.011034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cdfd8-8138-4d26-8f42-0a109143bd5d None admin] PATH_INFO: `/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/76cc0ae48b5a49c6a7a1c11a08c389e4/roles/29b8b950372848ffa9a75e778ee66b4c` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:41.025225 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20cdfd8-8138-4d26-8f42-0a109143bd5d None admin] RBAC: Authorizing `identity:create_grant(project_id=3f777cd6a0c744e594f8e29ab58cf8bd, user_id=76cc0ae48b5a49c6a7a1c11a08c389e4, role_id=29b8b950372848ffa9a75e778ee66b4c)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:41.026777 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20cdfd8-8138-4d26-8f42-0a109143bd5d None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:41.112730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 617/1232] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:56:40 2022] PUT /identity/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/76cc0ae48b5a49c6a7a1c11a08c389e4/roles/29b8b950372848ffa9a75e778ee66b4c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:56:43.156623 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35199580-15c1-4227-8030-2dca94cb5d4f None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:43.157231 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35199580-15c1-4227-8030-2dca94cb5d4f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:43.157231 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35199580-15c1-4227-8030-2dca94cb5d4f None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:43.158689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 616/1233] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:56:43.164425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b42b5c-23d4-4b50-8b1e-d8e4941bd165 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:43.164942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b42b5c-23d4-4b50-8b1e-d8e4941bd165 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:43.164942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b42b5c-23d4-4b50-8b1e-d8e4941bd165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:43.184691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-b9b42b5c-23d4-4b50-8b1e-d8e4941bd165 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:56:43.184691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b9b42b5c-23d4-4b50-8b1e-d8e4941bd165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:56:43.251348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 618/1234] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:56:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:56:43.276672 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35199580-15c1-4227-8030-2dca94cb5d4f None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:43.302729 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:43.379585 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35199580-15c1-4227-8030-2dca94cb5d4f None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:43.412404 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c589ba-8c0a-4621-b7c7-8f6e980371ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:43.413257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c589ba-8c0a-4621-b7c7-8f6e980371ee None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:43.413543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c589ba-8c0a-4621-b7c7-8f6e980371ee None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:43.413821 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c589ba-8c0a-4621-b7c7-8f6e980371ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:43.424361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c589ba-8c0a-4621-b7c7-8f6e980371ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:56:43.432019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-b9c589ba-8c0a-4621-b7c7-8f6e980371ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:56:43.432019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:43.432019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:43.432019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:43.432019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:43.432019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:43.432019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:43.432019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:43.432019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:43.432019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:43.432019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:43.432019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:43.432019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:56:43.432019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:56:43.432019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:56:43.432019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:56:43.433174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:56:43.433174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:56:43.433174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:56:43.433174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:56:43.433174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:43.433174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:43.433174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:43.433174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:43.433174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:43.433174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:43.433174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:43.433174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:43.433174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:43.433174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:43.433174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:43.433174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:43.434089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:43.434089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:43.434089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:43.434089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:43.434089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:56:43.434089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:56:43.434089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:56:43.434089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:56:43.434089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:56:43.434089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:56:43.434089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:56:43.434089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:56:43.434089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 617/1235] 10.222.0.83 () {56 vars in 1102 bytes} [Mon Jul 18 14:56:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:43.440504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b42b5c-23d4-4b50-8b1e-d8e4941bd165 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:43.488660 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:43.541172 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b42b5c-23d4-4b50-8b1e-d8e4941bd165 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:43.576135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae684f3a-c8a9-4e61-ad50-4698e3a7773b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:43.576836 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae684f3a-c8a9-4e61-ad50-4698e3a7773b None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:43.577123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae684f3a-c8a9-4e61-ad50-4698e3a7773b None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:43.577413 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae684f3a-c8a9-4e61-ad50-4698e3a7773b None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:43.578169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae684f3a-c8a9-4e61-ad50-4698e3a7773b None admin] RBAC: Adding query filter params (name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:43.578476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae684f3a-c8a9-4e61-ad50-4698e3a7773b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:43.580033 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae684f3a-c8a9-4e61-ad50-4698e3a7773b None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:43.588082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 619/1236] 10.222.0.83 () {56 vars in 1103 bytes} [Mon Jul 18 14:56:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:43.594045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c589ba-8c0a-4621-b7c7-8f6e980371ee None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:43.683564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:43.699217 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c589ba-8c0a-4621-b7c7-8f6e980371ee None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:43.734336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0f37d-fd0c-4e91-ad49-bee2197df543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:43.735272 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0f37d-fd0c-4e91-ad49-bee2197df543 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:43.735581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0f37d-fd0c-4e91-ad49-bee2197df543 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:43.735858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0f37d-fd0c-4e91-ad49-bee2197df543 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:43.746052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0f37d-fd0c-4e91-ad49-bee2197df543 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:43.747626 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0f37d-fd0c-4e91-ad49-bee2197df543 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:43.766392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-4bf0f37d-fd0c-4e91-ad49-bee2197df543 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:43.766392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:43.766392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:43.766392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:43.766392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:43.766392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:43.766392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:43.766392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:43.766392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:43.766392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:43.766392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:43.766392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:43.766392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:56:43.766392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:56:43.766392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:43.768146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:43.769232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:56:43.769232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:56:43.769232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:43.769232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:56:43.769232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 618/1237] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:56:43.774421 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae684f3a-c8a9-4e61-ad50-4698e3a7773b None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:43.876086 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae684f3a-c8a9-4e61-ad50-4698e3a7773b None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:43.885157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:43.909923 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e78a4b-f84f-4956-9340-002c2dd5591b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:43.910610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e78a4b-f84f-4956-9340-002c2dd5591b None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:43.910899 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e78a4b-f84f-4956-9340-002c2dd5591b None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:43.911175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e78a4b-f84f-4956-9340-002c2dd5591b None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:43.911921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e78a4b-f84f-4956-9340-002c2dd5591b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:43.912230 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e78a4b-f84f-4956-9340-002c2dd5591b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:43.913651 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e78a4b-f84f-4956-9340-002c2dd5591b None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:43.926840 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 620/1238] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:43.932576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0f37d-fd0c-4e91-ad49-bee2197df543 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:44.033792 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0f37d-fd0c-4e91-ad49-bee2197df543 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:44.069690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5a591-969f-43a4-9e13-4157bb5979e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:44.070506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5a591-969f-43a4-9e13-4157bb5979e3 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:44.070848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5a591-969f-43a4-9e13-4157bb5979e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:44.071148 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5a591-969f-43a4-9e13-4157bb5979e3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:44.073873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:44.078608 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5a591-969f-43a4-9e13-4157bb5979e3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:44.080977 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5a591-969f-43a4-9e13-4157bb5979e3 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:44.096590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-e5d5a591-969f-43a4-9e13-4157bb5979e3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:56:44.096590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:44.096590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:44.096590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:44.096590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:44.096590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:44.096590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:44.096590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:44.096590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:44.096590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:44.096590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:44.096590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:44.096590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:56:44.096590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:56:44.096590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:56:44.096590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:44.100459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:44.101533 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:44.101533 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:44.101533 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:56:44.101533 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:56:44.101533 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:56:44.101533 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:56:44.101533 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:56:44.101533 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:56:44.101533 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:56:44.101533 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:56:44.101533 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 619/1239] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:44.104257 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e78a4b-f84f-4956-9340-002c2dd5591b None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:44.207237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e78a4b-f84f-4956-9340-002c2dd5591b None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:44.244351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803165ec-eece-42f3-9d94-9a2f9ded052a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:44.245021 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803165ec-eece-42f3-9d94-9a2f9ded052a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:44.245308 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803165ec-eece-42f3-9d94-9a2f9ded052a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:44.245588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803165ec-eece-42f3-9d94-9a2f9ded052a None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:44.246427 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803165ec-eece-42f3-9d94-9a2f9ded052a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:44.246776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803165ec-eece-42f3-9d94-9a2f9ded052a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:44.248240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803165ec-eece-42f3-9d94-9a2f9ded052a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:44.268199 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 621/1240] 10.222.0.83 () {56 vars in 1143 bytes} [Mon Jul 18 14:56:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:44.274160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5a591-969f-43a4-9e13-4157bb5979e3 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:44.374758 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5a591-969f-43a4-9e13-4157bb5979e3 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:44.409714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a73d9d-0f6e-466b-9282-642f182a2011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:44.410365 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a73d9d-0f6e-466b-9282-642f182a2011 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:44.410679 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a73d9d-0f6e-466b-9282-642f182a2011 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:44.410955 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a73d9d-0f6e-466b-9282-642f182a2011 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:44.472038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:44.478749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a73d9d-0f6e-466b-9282-642f182a2011 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:44.480218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a73d9d-0f6e-466b-9282-642f182a2011 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:44.576854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 620/1241] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:56:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:56:44.583550 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803165ec-eece-42f3-9d94-9a2f9ded052a None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:44.611646 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:44.685403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803165ec-eece-42f3-9d94-9a2f9ded052a None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:44.720610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163cb66-0644-44cb-a777-ea0dd5b4596f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:44.721402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163cb66-0644-44cb-a777-ea0dd5b4596f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:44.721687 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163cb66-0644-44cb-a777-ea0dd5b4596f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:44.721965 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163cb66-0644-44cb-a777-ea0dd5b4596f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:44.732483 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163cb66-0644-44cb-a777-ea0dd5b4596f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:44.737709 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163cb66-0644-44cb-a777-ea0dd5b4596f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:44.753320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-d163cb66-0644-44cb-a777-ea0dd5b4596f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:44.753320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:44.753320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:44.753320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:44.753320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:44.753320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:44.753320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:44.753320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:44.753320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:44.753320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:44.753320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:44.753320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:44.753320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:56:44.753320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:56:44.753320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:44.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:44.757034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:56:44.757034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:56:44.757034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:44.757034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:56:44.757034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 622/1242] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:44.760539 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a73d9d-0f6e-466b-9282-642f182a2011 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:44.854864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:44.860606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a73d9d-0f6e-466b-9282-642f182a2011 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:44.894582 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04c82a-46a0-4188-9f99-324c2e5a2bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:44.895275 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04c82a-46a0-4188-9f99-324c2e5a2bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:44.895563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04c82a-46a0-4188-9f99-324c2e5a2bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:44.895843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04c82a-46a0-4188-9f99-324c2e5a2bf4 None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:44.896563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04c82a-46a0-4188-9f99-324c2e5a2bf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:44.896874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04c82a-46a0-4188-9f99-324c2e5a2bf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:44.898364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04c82a-46a0-4188-9f99-324c2e5a2bf4 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:44.910262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 621/1243] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:44.916249 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163cb66-0644-44cb-a777-ea0dd5b4596f None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:45.001027 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:45.021077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163cb66-0644-44cb-a777-ea0dd5b4596f None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:45.054921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf89b3-f0a2-4399-8b4b-d673d64e8396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:45.055739 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf89b3-f0a2-4399-8b4b-d673d64e8396 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:45.056042 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf89b3-f0a2-4399-8b4b-d673d64e8396 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:45.056321 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf89b3-f0a2-4399-8b4b-d673d64e8396 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:45.063004 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf89b3-f0a2-4399-8b4b-d673d64e8396 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:45.065003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf89b3-f0a2-4399-8b4b-d673d64e8396 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:45.078881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-6ccf89b3-f0a2-4399-8b4b-d673d64e8396 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:56:45.078881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:45.078881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:45.078881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:45.078881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:45.078881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:45.078881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:45.078881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:45.078881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:45.078881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:45.078881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:45.078881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:45.078881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:56:45.078881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:56:45.078881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:45.080693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:45.082283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:56:45.082283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:56:45.082283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:56:45.082283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:56:45.082283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:56:45.082283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:56:45.082283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:56:45.082283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:56:45.082283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 623/1244] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:56:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:45.086986 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04c82a-46a0-4188-9f99-324c2e5a2bf4 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:45.186560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04c82a-46a0-4188-9f99-324c2e5a2bf4 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:45.222653 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d313b-4a44-4366-a52f-a32df42e9925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:45.223389 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d313b-4a44-4366-a52f-a32df42e9925 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:45.223672 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d313b-4a44-4366-a52f-a32df42e9925 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:45.223948 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d313b-4a44-4366-a52f-a32df42e9925 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:45.224737 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d313b-4a44-4366-a52f-a32df42e9925 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:45.225053 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d313b-4a44-4366-a52f-a32df42e9925 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:45.226495 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d313b-4a44-4366-a52f-a32df42e9925 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:45.238777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 622/1245] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:56:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:45.245098 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf89b3-f0a2-4399-8b4b-d673d64e8396 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:45.354917 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf89b3-f0a2-4399-8b4b-d673d64e8396 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:45.388434 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de24a9-a162-4c09-8a93-fd5b3de080ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:45.389129 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de24a9-a162-4c09-8a93-fd5b3de080ef None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:45.389416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de24a9-a162-4c09-8a93-fd5b3de080ef None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:45.389695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de24a9-a162-4c09-8a93-fd5b3de080ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:45.390525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de24a9-a162-4c09-8a93-fd5b3de080ef None admin] RBAC: Adding query filter params (role.id=29b8b950372848ffa9a75e778ee66b4c, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=76cc0ae48b5a49c6a7a1c11a08c389e4) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:45.390868 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de24a9-a162-4c09-8a93-fd5b3de080ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:45.392394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de24a9-a162-4c09-8a93-fd5b3de080ef None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:45.398325 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 624/1246] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:56:45 2022] GET /identity/v3/role_assignments?user.id=76cc0ae48b5a49c6a7a1c11a08c389e4&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=29b8b950372848ffa9a75e778ee66b4c => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:47.558819 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c56e8-8561-4b9f-ac10-4fd107dcd9d5 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:47.559408 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c56e8-8561-4b9f-ac10-4fd107dcd9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:47.559408 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c56e8-8561-4b9f-ac10-4fd107dcd9d5 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:47.560746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 623/1247] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:56:47.566290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49e303-1d7e-4d76-a61f-fffe4f141038 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:47.566730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49e303-1d7e-4d76-a61f-fffe4f141038 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:47.566881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49e303-1d7e-4d76-a61f-fffe4f141038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:47.572032 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:47.584051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-ca49e303-1d7e-4d76-a61f-fffe4f141038 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:56:47.585242 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ca49e303-1d7e-4d76-a61f-fffe4f141038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:56:47.652689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 625/1248] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:56:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:56:47.678178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c56e8-8561-4b9f-ac10-4fd107dcd9d5 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:47.696528 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:47.787313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c56e8-8561-4b9f-ac10-4fd107dcd9d5 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:47.820876 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ffd62-37ae-48b9-88b7-1b62c3ca3490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:47.821757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ffd62-37ae-48b9-88b7-1b62c3ca3490 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:47.822210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ffd62-37ae-48b9-88b7-1b62c3ca3490 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:47.823237 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ffd62-37ae-48b9-88b7-1b62c3ca3490 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:47.833864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ffd62-37ae-48b9-88b7-1b62c3ca3490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:56:47.841534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-795ffd62-37ae-48b9-88b7-1b62c3ca3490 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:56:47.841534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:47.841534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:47.841534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:47.841534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:47.841534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:47.841534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:47.841534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:47.841534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:47.841534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:47.841534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:47.841534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:47.841534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:56:47.841534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:56:47.841534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:56:47.841534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:56:47.842746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:56:47.842746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:56:47.842746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:56:47.842746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:56:47.842746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:47.842746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:47.842746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:47.842746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:47.842746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:47.842746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:47.842746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:47.842746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:47.842746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:47.842746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:47.842746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:47.842746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:47.843854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:47.843854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:47.843854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:47.843854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:47.843854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:56:47.843854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:56:47.843854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:56:47.843854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:56:47.843854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:56:47.843854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:56:47.843854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:56:47.843854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:56:47.843854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 624/1249] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:56:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:47.850248 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49e303-1d7e-4d76-a61f-fffe4f141038 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:47.950982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49e303-1d7e-4d76-a61f-fffe4f141038 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:47.982895 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:47.988175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b809fa5-6f9a-41df-bd7e-b057ced883c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:47.988854 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b809fa5-6f9a-41df-bd7e-b057ced883c4 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:47.989145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b809fa5-6f9a-41df-bd7e-b057ced883c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:47.989427 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b809fa5-6f9a-41df-bd7e-b057ced883c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:47.990180 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b809fa5-6f9a-41df-bd7e-b057ced883c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:47.990500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b809fa5-6f9a-41df-bd7e-b057ced883c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:47.991919 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b809fa5-6f9a-41df-bd7e-b057ced883c4 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:48.000233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 626/1250] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:56:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:48.006194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ffd62-37ae-48b9-88b7-1b62c3ca3490 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:48.084942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:48.107611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ffd62-37ae-48b9-88b7-1b62c3ca3490 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:48.142844 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b8cc1-113a-482c-8c08-5604360d9cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:48.143725 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b8cc1-113a-482c-8c08-5604360d9cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:48.144013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b8cc1-113a-482c-8c08-5604360d9cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:48.144293 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b8cc1-113a-482c-8c08-5604360d9cb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:48.156355 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b8cc1-113a-482c-8c08-5604360d9cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:48.157872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b8cc1-113a-482c-8c08-5604360d9cb4 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:48.175021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-687b8cc1-113a-482c-8c08-5604360d9cb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:48.175021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:48.175021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:48.175021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:48.175021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:48.175021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:48.175021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:48.175021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:48.175021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:48.175021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:48.175021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:48.175021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:48.175021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:56:48.175021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:56:48.175021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:48.176143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:48.177493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:56:48.177493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:56:48.177493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:48.177493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:56:48.177493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 625/1251] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:48.182273 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b809fa5-6f9a-41df-bd7e-b057ced883c4 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:48.290796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b809fa5-6f9a-41df-bd7e-b057ced883c4 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:48.324322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c32c70-f62c-430e-b9cb-2a2a84cbb9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:48.324979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c32c70-f62c-430e-b9cb-2a2a84cbb9dd None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:48.325267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c32c70-f62c-430e-b9cb-2a2a84cbb9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:48.325546 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c32c70-f62c-430e-b9cb-2a2a84cbb9dd None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:48.326264 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c32c70-f62c-430e-b9cb-2a2a84cbb9dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:48.326595 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c32c70-f62c-430e-b9cb-2a2a84cbb9dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:48.327979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c32c70-f62c-430e-b9cb-2a2a84cbb9dd None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:48.340505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 627/1252] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:48.346222 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b8cc1-113a-482c-8c08-5604360d9cb4 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:48.447459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b8cc1-113a-482c-8c08-5604360d9cb4 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:48.474138 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:48.481747 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d0f5e-8b75-4356-aa55-e8329425c08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:48.482568 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d0f5e-8b75-4356-aa55-e8329425c08d None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:48.482859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d0f5e-8b75-4356-aa55-e8329425c08d None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:48.483139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d0f5e-8b75-4356-aa55-e8329425c08d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:48.491143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d0f5e-8b75-4356-aa55-e8329425c08d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:48.492593 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d0f5e-8b75-4356-aa55-e8329425c08d None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:48.507735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-714d0f5e-8b75-4356-aa55-e8329425c08d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:56:48.507735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:48.507735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:48.507735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:48.507735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:48.507735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:48.507735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:48.507735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:48.507735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:48.507735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:48.507735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:48.507735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:48.507735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:56:48.507735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:56:48.507735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:56:48.507735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:48.508942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:48.510134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:48.510134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:48.510134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:56:48.510134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:56:48.510134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:56:48.510134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:56:48.510134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:56:48.510134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:56:48.510134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:56:48.510134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:56:48.510134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 626/1253] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:48.515533 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c32c70-f62c-430e-b9cb-2a2a84cbb9dd None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:48.573124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:48.618978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c32c70-f62c-430e-b9cb-2a2a84cbb9dd None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:48.653689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdbaf0-6e31-426e-aa73-41a6e077d7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:48.654362 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdbaf0-6e31-426e-aa73-41a6e077d7ef None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:48.654675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdbaf0-6e31-426e-aa73-41a6e077d7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:48.654954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdbaf0-6e31-426e-aa73-41a6e077d7ef None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:48.655786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdbaf0-6e31-426e-aa73-41a6e077d7ef None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:48.656099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdbaf0-6e31-426e-aa73-41a6e077d7ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:48.658028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdbaf0-6e31-426e-aa73-41a6e077d7ef None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:48.675528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 628/1254] 10.222.0.83 () {56 vars in 1143 bytes} [Mon Jul 18 14:56:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:48.681460 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d0f5e-8b75-4356-aa55-e8329425c08d None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:48.785801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d0f5e-8b75-4356-aa55-e8329425c08d None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:48.820087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed037b20-56b0-4e03-94f4-a62c41e8e141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:48.820731 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed037b20-56b0-4e03-94f4-a62c41e8e141 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:48.821029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed037b20-56b0-4e03-94f4-a62c41e8e141 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:48.821310 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed037b20-56b0-4e03-94f4-a62c41e8e141 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:48.878834 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:48.891069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed037b20-56b0-4e03-94f4-a62c41e8e141 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:48.893551 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed037b20-56b0-4e03-94f4-a62c41e8e141 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:48.992491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 627/1255] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:56:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:56:48.998323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdbaf0-6e31-426e-aa73-41a6e077d7ef None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:49.099350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdbaf0-6e31-426e-aa73-41a6e077d7ef None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:49.112285 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:49.133760 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659471b3-3440-4a5c-b70d-5ae12bf53e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:49.134481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659471b3-3440-4a5c-b70d-5ae12bf53e7b None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:49.134797 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659471b3-3440-4a5c-b70d-5ae12bf53e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:49.135075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659471b3-3440-4a5c-b70d-5ae12bf53e7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:49.146807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659471b3-3440-4a5c-b70d-5ae12bf53e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:49.148269 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659471b3-3440-4a5c-b70d-5ae12bf53e7b None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:49.166082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-659471b3-3440-4a5c-b70d-5ae12bf53e7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:49.166082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:49.166082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:49.166082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:49.166082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:49.166082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:49.166082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:49.166082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:49.166082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:49.166082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:49.166082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:49.166082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:49.166082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:56:49.166082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:56:49.166082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:49.167805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:49.168737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:56:49.168737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:56:49.168737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:49.168737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:56:49.168737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 629/1256] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:49.173239 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed037b20-56b0-4e03-94f4-a62c41e8e141 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:49.263882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:49.275876 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed037b20-56b0-4e03-94f4-a62c41e8e141 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:49.312955 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97724446-c7e3-44b1-a75b-56811af557bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:49.313628 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97724446-c7e3-44b1-a75b-56811af557bf None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:49.313950 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97724446-c7e3-44b1-a75b-56811af557bf None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:49.314237 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97724446-c7e3-44b1-a75b-56811af557bf None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:49.315671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97724446-c7e3-44b1-a75b-56811af557bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:49.315982 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97724446-c7e3-44b1-a75b-56811af557bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:49.317412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97724446-c7e3-44b1-a75b-56811af557bf None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:49.328978 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 628/1257] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:49.335077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659471b3-3440-4a5c-b70d-5ae12bf53e7b None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:49.437921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659471b3-3440-4a5c-b70d-5ae12bf53e7b None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:49.473626 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5802708-0d26-4478-af8d-544e3f1534e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:49.474474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5802708-0d26-4478-af8d-544e3f1534e6 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:49.474792 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5802708-0d26-4478-af8d-544e3f1534e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:49.475074 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5802708-0d26-4478-af8d-544e3f1534e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:49.482082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5802708-0d26-4478-af8d-544e3f1534e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:49.483680 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5802708-0d26-4478-af8d-544e3f1534e6 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:49.498323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-f5802708-0d26-4478-af8d-544e3f1534e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:56:49.498323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:49.498323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:49.498323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:49.498323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:49.498323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:49.498323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:49.498323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:49.498323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:49.498323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:49.498323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:49.498323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:49.498323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:56:49.498323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:56:49.498323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:49.499420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:49.500465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:56:49.500465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:56:49.500465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:56:49.500465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:56:49.500465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:56:49.500465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:56:49.500465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:56:49.500465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:56:49.500465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 630/1258] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:56:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:49.506483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97724446-c7e3-44b1-a75b-56811af557bf None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:49.608853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97724446-c7e3-44b1-a75b-56811af557bf None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:49.642710 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f15d1-fee9-40b6-8d44-febbb932f879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:49.643520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f15d1-fee9-40b6-8d44-febbb932f879 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:49.643809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f15d1-fee9-40b6-8d44-febbb932f879 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:49.644091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f15d1-fee9-40b6-8d44-febbb932f879 None admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:49.644878 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f15d1-fee9-40b6-8d44-febbb932f879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:49.645189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f15d1-fee9-40b6-8d44-febbb932f879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:49.646656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f15d1-fee9-40b6-8d44-febbb932f879 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:49.658942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 629/1259] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:56:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:49.665267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5802708-0d26-4478-af8d-544e3f1534e6 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:49.692884 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:49.773927 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5802708-0d26-4478-af8d-544e3f1534e6 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:49.809613 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77015cbd-e9ec-4051-9275-448837d2962a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:49.810315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77015cbd-e9ec-4051-9275-448837d2962a None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:49.810627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77015cbd-e9ec-4051-9275-448837d2962a None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:49.810904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77015cbd-e9ec-4051-9275-448837d2962a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:49.811725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77015cbd-e9ec-4051-9275-448837d2962a None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=76cc0ae48b5a49c6a7a1c11a08c389e4) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:49.812038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77015cbd-e9ec-4051-9275-448837d2962a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:49.813501 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77015cbd-e9ec-4051-9275-448837d2962a None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:49.820056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 631/1260] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:56:49 2022] GET /identity/v3/role_assignments?user.id=76cc0ae48b5a49c6a7a1c11a08c389e4&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=5891920310b54787bd1152d736d2a3f1 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:51.900504 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d455d54-c765-47a4-a7c0-b0291a894cda None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:51.901096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d455d54-c765-47a4-a7c0-b0291a894cda None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:51.901096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d455d54-c765-47a4-a7c0-b0291a894cda None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:51.902420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 630/1261] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:56:51.907593 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae48522-5bbf-4f9b-aac6-277baa6932e1 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:51.909157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae48522-5bbf-4f9b-aac6-277baa6932e1 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:51.909157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae48522-5bbf-4f9b-aac6-277baa6932e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:51.926916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-6ae48522-5bbf-4f9b-aac6-277baa6932e1 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:56:51.928197 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-6ae48522-5bbf-4f9b-aac6-277baa6932e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:56:51.997576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 632/1262] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:56:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:56:52.020984 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d455d54-c765-47a4-a7c0-b0291a894cda None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:52.031861 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:52.123582 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d455d54-c765-47a4-a7c0-b0291a894cda None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:52.160346 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c980f8a-fc46-40e1-90d0-a53b218acbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:52.161102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c980f8a-fc46-40e1-90d0-a53b218acbda None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:52.161384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c980f8a-fc46-40e1-90d0-a53b218acbda None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:52.161657 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c980f8a-fc46-40e1-90d0-a53b218acbda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:52.171605 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c980f8a-fc46-40e1-90d0-a53b218acbda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:56:52.180441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-4c980f8a-fc46-40e1-90d0-a53b218acbda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:56:52.180441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:52.180441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:52.180441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:52.180441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:52.180441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:52.180441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:52.180441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:52.180441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:52.180441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:52.180441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:52.180441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:52.180441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:56:52.180441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:56:52.180441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:56:52.180441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:56:52.181685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:56:52.181685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:56:52.181685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:56:52.181685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:56:52.181685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:52.181685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:52.181685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:52.181685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:52.181685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:52.181685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:52.181685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:52.181685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:52.181685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:52.181685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:52.181685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:52.181685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:52.183045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:52.183045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:52.183045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:52.183045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:52.183045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:56:52.183045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:56:52.183045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:56:52.183045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:56:52.183045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:56:52.183045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:56:52.183045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:56:52.183045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:56:52.183045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 631/1263] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:56:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:52.187785 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae48522-5bbf-4f9b-aac6-277baa6932e1 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:52.206950 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:52.288217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae48522-5bbf-4f9b-aac6-277baa6932e1 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:52.324139 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3ea3c-b01a-4f3a-9811-0312d0e027fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:52.324808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3ea3c-b01a-4f3a-9811-0312d0e027fe None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:52.325094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3ea3c-b01a-4f3a-9811-0312d0e027fe None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:52.325390 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3ea3c-b01a-4f3a-9811-0312d0e027fe None admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:52.326135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3ea3c-b01a-4f3a-9811-0312d0e027fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:52.326439 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3ea3c-b01a-4f3a-9811-0312d0e027fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:52.327858 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3ea3c-b01a-4f3a-9811-0312d0e027fe None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:52.335924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 633/1264] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:56:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:52.341773 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c980f8a-fc46-40e1-90d0-a53b218acbda None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:52.412795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:52.446892 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c980f8a-fc46-40e1-90d0-a53b218acbda None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:52.480898 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa0fb1-93a3-47a0-837f-9e3fda94d7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:52.481703 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa0fb1-93a3-47a0-837f-9e3fda94d7ab None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:52.481993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa0fb1-93a3-47a0-837f-9e3fda94d7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:52.482270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa0fb1-93a3-47a0-837f-9e3fda94d7ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:52.494014 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa0fb1-93a3-47a0-837f-9e3fda94d7ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:52.498023 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa0fb1-93a3-47a0-837f-9e3fda94d7ab None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:52.513776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-28aa0fb1-93a3-47a0-837f-9e3fda94d7ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:52.513776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:52.513776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:52.513776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:52.513776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:52.513776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:52.513776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:52.513776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:52.513776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:52.513776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:52.513776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:52.513776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:52.513776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:56:52.513776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:56:52.513776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:52.514893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:52.516035 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:56:52.516035 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:56:52.516035 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:52.516035 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:56:52.516035 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 632/1265] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:52.521214 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3ea3c-b01a-4f3a-9811-0312d0e027fe None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:52.602865 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:52.622526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3ea3c-b01a-4f3a-9811-0312d0e027fe None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:52.657348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f980ab5-9fa3-4fa8-a89f-b3253939fb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:52.658526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f980ab5-9fa3-4fa8-a89f-b3253939fb74 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:52.658840 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f980ab5-9fa3-4fa8-a89f-b3253939fb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:52.659116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f980ab5-9fa3-4fa8-a89f-b3253939fb74 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:52.659835 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f980ab5-9fa3-4fa8-a89f-b3253939fb74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:52.660135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f980ab5-9fa3-4fa8-a89f-b3253939fb74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:52.661522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f980ab5-9fa3-4fa8-a89f-b3253939fb74 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:52.672682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 634/1266] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:52.678802 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa0fb1-93a3-47a0-837f-9e3fda94d7ab None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:52.779927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa0fb1-93a3-47a0-837f-9e3fda94d7ab None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:52.799968 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:52.813538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e85277-d37f-4283-b780-d0e6ee3252f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:52.814952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e85277-d37f-4283-b780-d0e6ee3252f3 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:52.815320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e85277-d37f-4283-b780-d0e6ee3252f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:52.815599 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e85277-d37f-4283-b780-d0e6ee3252f3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:52.823113 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e85277-d37f-4283-b780-d0e6ee3252f3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:52.824563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e85277-d37f-4283-b780-d0e6ee3252f3 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:52.839642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-39e85277-d37f-4283-b780-d0e6ee3252f3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:56:52.839642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:52.839642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:52.839642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:52.839642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:52.839642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:52.839642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:52.839642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:52.839642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:52.839642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:52.839642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:52.839642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:52.839642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:56:52.839642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:56:52.839642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:56:52.839642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:52.840879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:52.841895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:52.841895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:52.841895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:56:52.841895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:56:52.841895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:56:52.841895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:56:52.841895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:56:52.841895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:56:52.841895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:56:52.841895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:56:52.841895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 633/1267] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:52.847440 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f980ab5-9fa3-4fa8-a89f-b3253939fb74 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:52.955154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f980ab5-9fa3-4fa8-a89f-b3253939fb74 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:52.989047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9accab-fe44-4743-8973-a33bb5817ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:52.989723 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9accab-fe44-4743-8973-a33bb5817ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:52.990009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9accab-fe44-4743-8973-a33bb5817ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:52.990288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9accab-fe44-4743-8973-a33bb5817ef0 None admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:52.991166 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9accab-fe44-4743-8973-a33bb5817ef0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:52.991528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9accab-fe44-4743-8973-a33bb5817ef0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:52.992991 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9accab-fe44-4743-8973-a33bb5817ef0 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:53.002885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:53.011125 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 635/1268] 10.222.0.83 () {56 vars in 1143 bytes} [Mon Jul 18 14:56:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:53.017150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e85277-d37f-4283-b780-d0e6ee3252f3 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:53.119040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e85277-d37f-4283-b780-d0e6ee3252f3 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:53.153459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9c754-75bc-4975-a534-21435ad14691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:53.154245 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9c754-75bc-4975-a534-21435ad14691 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:53.154529 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9c754-75bc-4975-a534-21435ad14691 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:53.154837 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9c754-75bc-4975-a534-21435ad14691 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:53.166389 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9c754-75bc-4975-a534-21435ad14691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:53.167885 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9c754-75bc-4975-a534-21435ad14691 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:53.190888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-21b9c754-75bc-4975-a534-21435ad14691 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:53.190888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:53.190888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:53.190888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:53.190888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:53.190888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:53.190888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:53.190888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:53.190888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:53.190888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:53.190888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:53.190888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:53.190888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:56:53.190888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:56:53.190888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:53.192022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:53.193151 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:56:53.193151 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:56:53.193151 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:53.193151 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:56:53.193151 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 634/1269] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:53.193664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9accab-fe44-4743-8973-a33bb5817ef0 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:53.296288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9accab-fe44-4743-8973-a33bb5817ef0 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:53.331464 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6208130-5216-4650-8984-a1c44a23b1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:53.332128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6208130-5216-4650-8984-a1c44a23b1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:53.332419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6208130-5216-4650-8984-a1c44a23b1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:53.332702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6208130-5216-4650-8984-a1c44a23b1d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:53.333425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6208130-5216-4650-8984-a1c44a23b1d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:53.333790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6208130-5216-4650-8984-a1c44a23b1d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:53.335213 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6208130-5216-4650-8984-a1c44a23b1d0 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:53.346624 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 636/1270] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:56:53.353023 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9c754-75bc-4975-a534-21435ad14691 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:53.385606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:53.453427 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9c754-75bc-4975-a534-21435ad14691 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:53.487977 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ee48e-cf47-4e9f-b5a3-38e9bbb22690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:53.488863 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ee48e-cf47-4e9f-b5a3-38e9bbb22690 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:53.489151 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ee48e-cf47-4e9f-b5a3-38e9bbb22690 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:53.489432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ee48e-cf47-4e9f-b5a3-38e9bbb22690 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:53.496257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ee48e-cf47-4e9f-b5a3-38e9bbb22690 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:53.497750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ee48e-cf47-4e9f-b5a3-38e9bbb22690 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:53.512888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-284ee48e-cf47-4e9f-b5a3-38e9bbb22690 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:56:53.512888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:53.512888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:53.512888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:53.512888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:53.512888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:53.512888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:53.512888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:53.512888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:53.512888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:53.512888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:53.512888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:53.512888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:56:53.512888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:56:53.512888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:53.514087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:53.521302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:56:53.521302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:56:53.521302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:56:53.521302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:56:53.521302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:56:53.521302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:56:53.521302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:56:53.521302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:56:53.521302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 635/1271] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:56:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:53.521952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6208130-5216-4650-8984-a1c44a23b1d0 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:53.581204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:53.623031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6208130-5216-4650-8984-a1c44a23b1d0 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:53.657394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80fa31-a1f2-48ef-92cb-bd0d6254cb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:53.658087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80fa31-a1f2-48ef-92cb-bd0d6254cb31 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:53.659069 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80fa31-a1f2-48ef-92cb-bd0d6254cb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:53.659069 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80fa31-a1f2-48ef-92cb-bd0d6254cb31 None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:53.659614 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80fa31-a1f2-48ef-92cb-bd0d6254cb31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:53.659922 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80fa31-a1f2-48ef-92cb-bd0d6254cb31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:53.661379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80fa31-a1f2-48ef-92cb-bd0d6254cb31 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:53.673373 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 637/1272] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:56:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:53.680399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ee48e-cf47-4e9f-b5a3-38e9bbb22690 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:53.776074 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:53.786811 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ee48e-cf47-4e9f-b5a3-38e9bbb22690 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:53.822595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3bcb0-75b7-480f-95b9-f182f2e1a7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:53.823482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3bcb0-75b7-480f-95b9-f182f2e1a7d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:53.823762 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3bcb0-75b7-480f-95b9-f182f2e1a7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:53.824032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3bcb0-75b7-480f-95b9-f182f2e1a7d7 None admin] PATH_INFO: `/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/76cc0ae48b5a49c6a7a1c11a08c389e4/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:53.825957 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3bcb0-75b7-480f-95b9-f182f2e1a7d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f777cd6a0c744e594f8e29ab58cf8bd, user_id=76cc0ae48b5a49c6a7a1c11a08c389e4, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:53.827424 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3bcb0-75b7-480f-95b9-f182f2e1a7d7 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:53.901526 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 636/1273] 10.222.0.83 () {58 vars in 1439 bytes} [Mon Jul 18 14:56:53 2022] PUT /identity/v3/projects/3f777cd6a0c744e594f8e29ab58cf8bd/users/76cc0ae48b5a49c6a7a1c11a08c389e4/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:56:55.954393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e1cd4-a1bc-40c3-995c-36ffaa474613 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:55.955369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e1cd4-a1bc-40c3-995c-36ffaa474613 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:55.955369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e1cd4-a1bc-40c3-995c-36ffaa474613 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:55.956470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 638/1274] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:56:55.961895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9dbfb-74cc-442a-b3a3-43ff4f65340b None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:55.962322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9dbfb-74cc-442a-b3a3-43ff4f65340b None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:55.966616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9dbfb-74cc-442a-b3a3-43ff4f65340b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:55.980662 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b9f9dbfb-74cc-442a-b3a3-43ff4f65340b None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:56:55.981860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b9f9dbfb-74cc-442a-b3a3-43ff4f65340b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:56:56.052048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 637/1275] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:56:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:56:56.076777 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e1cd4-a1bc-40c3-995c-36ffaa474613 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:56.180757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e1cd4-a1bc-40c3-995c-36ffaa474613 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:56.198774 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:56.215370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76588f-8879-4238-ae9f-b2e3efff5bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:56.216171 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76588f-8879-4238-ae9f-b2e3efff5bcb None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:56.216460 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76588f-8879-4238-ae9f-b2e3efff5bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:56.216741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76588f-8879-4238-ae9f-b2e3efff5bcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:56.227086 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76588f-8879-4238-ae9f-b2e3efff5bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:56:56.234887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-ae76588f-8879-4238-ae9f-b2e3efff5bcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:56:56.234887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:56.234887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:56.234887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:56.234887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:56.234887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:56.234887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:56.234887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:56.234887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:56.234887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:56.234887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:56.234887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:56.234887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:56:56.234887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:56:56.234887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:56:56.234887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:56:56.249425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:56:56.249425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:56:56.249425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:56:56.249425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:56:56.249425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:56.249425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:56.249425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:56.249425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:56.249425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:56.249425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:56.249425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:56.249425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:56.249425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:56.249425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:56.249425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:56.249425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:56.250571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:56.250571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:56.250571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:56.250571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:56.250571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:56:56.250571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:56:56.250571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:56:56.250571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:56:56.250571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:56:56.250571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:56:56.250571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:56:56.250571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:56:56.250571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 639/1276] 10.222.0.83 () {56 vars in 1096 bytes} [Mon Jul 18 14:56:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:56.252115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9dbfb-74cc-442a-b3a3-43ff4f65340b None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:56.326326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:56.344278 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9dbfb-74cc-442a-b3a3-43ff4f65340b None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:56.383515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4890608-1ed3-4009-8115-abb48f67a8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:56.384177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4890608-1ed3-4009-8115-abb48f67a8ec None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:56.384462 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4890608-1ed3-4009-8115-abb48f67a8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:56.384740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4890608-1ed3-4009-8115-abb48f67a8ec None admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:56.385484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4890608-1ed3-4009-8115-abb48f67a8ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:56.385791 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4890608-1ed3-4009-8115-abb48f67a8ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:56.387206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4890608-1ed3-4009-8115-abb48f67a8ec None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:56.395545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 638/1277] 10.222.0.83 () {56 vars in 1099 bytes} [Mon Jul 18 14:56:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:56.401679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76588f-8879-4238-ae9f-b2e3efff5bcb None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:56.507831 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76588f-8879-4238-ae9f-b2e3efff5bcb None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:56.541342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3de460-70bb-45c2-9ffb-e2c213bb6336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:56.542137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3de460-70bb-45c2-9ffb-e2c213bb6336 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:56.542426 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3de460-70bb-45c2-9ffb-e2c213bb6336 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:56.542737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3de460-70bb-45c2-9ffb-e2c213bb6336 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:56.554045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3de460-70bb-45c2-9ffb-e2c213bb6336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:56.555546 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3de460-70bb-45c2-9ffb-e2c213bb6336 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:56.572741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-ab3de460-70bb-45c2-9ffb-e2c213bb6336 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:56.572741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:56.572741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:56.572741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:56.572741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:56.572741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:56.572741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:56.572741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:56.572741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:56.572741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:56.572741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:56.572741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:56.572741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:56:56.572741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:56:56.572741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:56.576486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:56.577702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:56:56.577702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:56:56.577702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:56.577702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:56:56.577702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 640/1278] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:56.580485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4890608-1ed3-4009-8115-abb48f67a8ec None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:56.685359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4890608-1ed3-4009-8115-abb48f67a8ec None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:56.718745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270d03e-f03d-49e8-a3e0-34ef1af2b32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:56.719398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270d03e-f03d-49e8-a3e0-34ef1af2b32b None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:56.719681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270d03e-f03d-49e8-a3e0-34ef1af2b32b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:56.719958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270d03e-f03d-49e8-a3e0-34ef1af2b32b None admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:56.721253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270d03e-f03d-49e8-a3e0-34ef1af2b32b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:56.721818 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270d03e-f03d-49e8-a3e0-34ef1af2b32b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:56.723305 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270d03e-f03d-49e8-a3e0-34ef1af2b32b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:56.735821 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 639/1279] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:56.741670 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3de460-70bb-45c2-9ffb-e2c213bb6336 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:56.774553 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:56.843344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3de460-70bb-45c2-9ffb-e2c213bb6336 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:56.880175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe028e-ea0e-4fd3-9c12-ccb13ecf2bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:56.880985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe028e-ea0e-4fd3-9c12-ccb13ecf2bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:56.881264 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe028e-ea0e-4fd3-9c12-ccb13ecf2bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:56.881540 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe028e-ea0e-4fd3-9c12-ccb13ecf2bf5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:56.888602 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe028e-ea0e-4fd3-9c12-ccb13ecf2bf5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:56.890104 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe028e-ea0e-4fd3-9c12-ccb13ecf2bf5 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:56.906555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-75fe028e-ea0e-4fd3-9c12-ccb13ecf2bf5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:56:56.906555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:56.906555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:56.906555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:56.906555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:56.906555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:56.906555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:56.906555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:56.906555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:56.906555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:56.906555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:56.906555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:56.906555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:56:56.906555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:56:56.906555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:56:56.906555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:56.908304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:56.909762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:56.909762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:56.909762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:56:56.909762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:56:56.909762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:56:56.909762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:56:56.909762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:56:56.909762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:56:56.909762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:56:56.909762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:56:56.909762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 641/1280] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:56.914394 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270d03e-f03d-49e8-a3e0-34ef1af2b32b None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:56.918423 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:57.015772 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270d03e-f03d-49e8-a3e0-34ef1af2b32b None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:57.053158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5b3bb-b205-4aff-bfe9-2c12c2f6ba28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:57.053769 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5b3bb-b205-4aff-bfe9-2c12c2f6ba28 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:57.054090 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5b3bb-b205-4aff-bfe9-2c12c2f6ba28 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:57.054395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5b3bb-b205-4aff-bfe9-2c12c2f6ba28 None admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:57.055414 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5b3bb-b205-4aff-bfe9-2c12c2f6ba28 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:57.055760 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5b3bb-b205-4aff-bfe9-2c12c2f6ba28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:57.057210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5b3bb-b205-4aff-bfe9-2c12c2f6ba28 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:57.075934 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 640/1281] 10.222.0.83 () {56 vars in 1143 bytes} [Mon Jul 18 14:56:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:57.082794 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe028e-ea0e-4fd3-9c12-ccb13ecf2bf5 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:57.180476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:57.190831 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe028e-ea0e-4fd3-9c12-ccb13ecf2bf5 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:57.225514 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fec150-b62c-44f4-a18f-956d1c589519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:57.226166 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fec150-b62c-44f4-a18f-956d1c589519 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:57.226452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fec150-b62c-44f4-a18f-956d1c589519 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:57.226763 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fec150-b62c-44f4-a18f-956d1c589519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:57.294920 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fec150-b62c-44f4-a18f-956d1c589519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:57.296401 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fec150-b62c-44f4-a18f-956d1c589519 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:57.389814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:57.400903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 642/1282] 10.222.0.83 () {58 vars in 1282 bytes} [Mon Jul 18 14:56:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:56:57.406966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5b3bb-b205-4aff-bfe9-2c12c2f6ba28 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:57.472650 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:57.512048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5b3bb-b205-4aff-bfe9-2c12c2f6ba28 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:57.545462 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0155d30-5414-4ec8-8f5d-fba310647dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:57.546250 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0155d30-5414-4ec8-8f5d-fba310647dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:57.546560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0155d30-5414-4ec8-8f5d-fba310647dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:57.546930 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0155d30-5414-4ec8-8f5d-fba310647dd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:57.557555 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0155d30-5414-4ec8-8f5d-fba310647dd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:57.559053 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0155d30-5414-4ec8-8f5d-fba310647dd8 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:57.577085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-b0155d30-5414-4ec8-8f5d-fba310647dd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:57.577085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:57.577085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:57.577085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:57.577085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:57.577085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:57.577085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:57.577085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:57.577085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:57.577085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:57.577085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:57.577085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:57.577085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:56:57.577085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:56:57.577085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:57.578400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:57.581606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:56:57.581606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:56:57.581606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:56:57.581606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:56:57.581606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 641/1283] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 14:56:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:56:57.585324 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fec150-b62c-44f4-a18f-956d1c589519 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:57.686110 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fec150-b62c-44f4-a18f-956d1c589519 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:57.723516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412e77e-0490-4531-9919-2630cfb275d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:57.724186 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412e77e-0490-4531-9919-2630cfb275d4 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:57.724482 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412e77e-0490-4531-9919-2630cfb275d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:57.724761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412e77e-0490-4531-9919-2630cfb275d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:57.725488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412e77e-0490-4531-9919-2630cfb275d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:57.725797 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412e77e-0490-4531-9919-2630cfb275d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:57.727224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412e77e-0490-4531-9919-2630cfb275d4 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:57.739461 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 643/1284] 10.222.0.83 () {56 vars in 1109 bytes} [Mon Jul 18 14:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:57.745419 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0155d30-5414-4ec8-8f5d-fba310647dd8 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:57.847555 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0155d30-5414-4ec8-8f5d-fba310647dd8 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:57.863532 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:57.883647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d3417-68ed-46af-85f7-9fee2ca2c172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:57.884454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d3417-68ed-46af-85f7-9fee2ca2c172 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:57.884744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d3417-68ed-46af-85f7-9fee2ca2c172 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:57.885026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d3417-68ed-46af-85f7-9fee2ca2c172 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:57.891975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d3417-68ed-46af-85f7-9fee2ca2c172 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:57.893434 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d3417-68ed-46af-85f7-9fee2ca2c172 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:57.908264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application [None req-012d3417-68ed-46af-85f7-9fee2ca2c172 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:56:57.908264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:56:57.908264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:56:57.908264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:56:57.908264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:56:57.908264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:56:57.908264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:56:57.908264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:56:57.908264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:56:57.908264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:56:57.908264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:56:57.908264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:56:57.908264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:56:57.908264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:56:57.908264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application with Lock( Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:56:57.909322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:56:57.910483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:56:57.910483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:56:57.910483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:56:57.910483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:56:57.910483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:56:57.910483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:56:57.910483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:56:57.910483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: ERROR keystone.server.flask.application Jul 18 14:56:57.910483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 642/1285] 10.222.0.83 () {56 vars in 1111 bytes} [Mon Jul 18 14:56:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:56:57.915737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412e77e-0490-4531-9919-2630cfb275d4 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:57.968315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:56:58.021011 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412e77e-0490-4531-9919-2630cfb275d4 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:58.056764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcded390-80a1-49db-b66a-9629d27a193f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:58.057555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcded390-80a1-49db-b66a-9629d27a193f None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:58.057933 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcded390-80a1-49db-b66a-9629d27a193f None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:58.058311 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcded390-80a1-49db-b66a-9629d27a193f None admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:58.059237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcded390-80a1-49db-b66a-9629d27a193f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:58.059648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcded390-80a1-49db-b66a-9629d27a193f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:58.061892 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcded390-80a1-49db-b66a-9629d27a193f None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:58.074829 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 644/1286] 10.222.0.83 () {56 vars in 1148 bytes} [Mon Jul 18 14:56:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:56:58.081214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d3417-68ed-46af-85f7-9fee2ca2c172 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:56:58.183972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d3417-68ed-46af-85f7-9fee2ca2c172 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:56:58.217795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68200c8e-4a41-4040-8a1c-8a244866ad78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:56:58.218489 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68200c8e-4a41-4040-8a1c-8a244866ad78 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:56:58.218766 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68200c8e-4a41-4040-8a1c-8a244866ad78 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:56:58.219050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68200c8e-4a41-4040-8a1c-8a244866ad78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:56:58.219882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68200c8e-4a41-4040-8a1c-8a244866ad78 None admin] RBAC: Adding query filter params (role.id=5891920310b54787bd1152d736d2a3f1, scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd, user.id=76cc0ae48b5a49c6a7a1c11a08c389e4) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:56:58.221011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68200c8e-4a41-4040-8a1c-8a244866ad78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:56:58.222801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68200c8e-4a41-4040-8a1c-8a244866ad78 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:56:58.228688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 643/1287] 10.222.0.83 () {56 vars in 1374 bytes} [Mon Jul 18 14:56:58 2022] GET /identity/v3/role_assignments?user.id=76cc0ae48b5a49c6a7a1c11a08c389e4&scope.project.id=3f777cd6a0c744e594f8e29ab58cf8bd&role.id=5891920310b54787bd1152d736d2a3f1 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:57:00.331015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984944ec-1775-4037-b4b1-e317a657addd None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:00.331612 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984944ec-1775-4037-b4b1-e317a657addd None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:00.331612 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984944ec-1775-4037-b4b1-e317a657addd None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:00.333028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 645/1288] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:57:00.338307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b3a7e-d685-41ed-8ac3-fecc951bd74d None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:00.338852 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b3a7e-d685-41ed-8ac3-fecc951bd74d None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:00.339139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b3a7e-d685-41ed-8ac3-fecc951bd74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:00.357714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b03b3a7e-d685-41ed-8ac3-fecc951bd74d None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:57:00.358987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b03b3a7e-d685-41ed-8ac3-fecc951bd74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:57:00.389804 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:57:00.430016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 644/1289] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:57:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:57:00.452942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984944ec-1775-4037-b4b1-e317a657addd None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:57:00.558641 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984944ec-1775-4037-b4b1-e317a657addd None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:57:00.569826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:57:00.593911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340b332-2a42-4c74-bc89-3e57c6f04fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:57:00.594667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340b332-2a42-4c74-bc89-3e57c6f04fae None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:00.594954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340b332-2a42-4c74-bc89-3e57c6f04fae None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:00.596128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340b332-2a42-4c74-bc89-3e57c6f04fae None admin] PATH_INFO: `/v3/services/placement` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:00.596755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340b332-2a42-4c74-bc89-3e57c6f04fae None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:57:00.598212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340b332-2a42-4c74-bc89-3e57c6f04fae None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-e340b332-2a42-4c74-bc89-3e57c6f04fae None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:57:00.610341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:57:00.612111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:57:00.612111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:57:00.612111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:57:00.612111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:57:00.612111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:57:00.612111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:57:00.612111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:57:00.612111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:57:00.612111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:57:00.612111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:57:00.612111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:57:00.612111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:57:00.612111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:57:00.612111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:57:00.612111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:57:00.612111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:57:00.613016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:57:00.613016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:57:00.613016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:57:00.613016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:57:00.613016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:57:00.613016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:57:00.613016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:57:00.613016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:57:00.613016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:57:00.613016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:57:00.613016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:57:00.613016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:57:00.613016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:57:00.613016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:57:00.613016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 646/1290] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Jul 18 14:57:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:57:00.617791 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b3a7e-d685-41ed-8ac3-fecc951bd74d None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:57:00.717104 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b3a7e-d685-41ed-8ac3-fecc951bd74d None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:57:00.752390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7515c67-291f-4a35-8fc5-f6f248c214cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:57:00.753051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7515c67-291f-4a35-8fc5-f6f248c214cb None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:00.753338 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7515c67-291f-4a35-8fc5-f6f248c214cb None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:00.753627 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7515c67-291f-4a35-8fc5-f6f248c214cb None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:00.754335 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7515c67-291f-4a35-8fc5-f6f248c214cb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:57:00.754676 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7515c67-291f-4a35-8fc5-f6f248c214cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:57:00.756051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7515c67-291f-4a35-8fc5-f6f248c214cb None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:57:00.760855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 645/1291] 10.222.0.83 () {56 vars in 1116 bytes} [Mon Jul 18 14:57:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:57:00.767572 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340b332-2a42-4c74-bc89-3e57c6f04fae None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:57:00.871302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340b332-2a42-4c74-bc89-3e57c6f04fae None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:57:00.906189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d1626-d125-4829-8695-f797935b205e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:57:00.906880 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d1626-d125-4829-8695-f797935b205e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:00.907168 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d1626-d125-4829-8695-f797935b205e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:00.907464 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d1626-d125-4829-8695-f797935b205e None admin] PATH_INFO: `/v3/services` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:00.908243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d1626-d125-4829-8695-f797935b205e None admin] RBAC: Adding query filter params (type=placement) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:57:00.908550 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d1626-d125-4829-8695-f797935b205e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:57:00.909943 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d1626-d125-4829-8695-f797935b205e None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:57:00.914549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 647/1292] 10.222.0.83 () {56 vars in 1116 bytes} [Mon Jul 18 14:57:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:57:02.992330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e2ba1-e50c-460b-84b8-f8da3c9c4693 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:02.992895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e2ba1-e50c-460b-84b8-f8da3c9c4693 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:02.992895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e2ba1-e50c-460b-84b8-f8da3c9c4693 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:02.994319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 646/1293] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:57:02.999641 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4418e5-209a-498f-b513-d053a1773cb7 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:03.000081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4418e5-209a-498f-b513-d053a1773cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:03.000178 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4418e5-209a-498f-b513-d053a1773cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:03.019318 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-4b4418e5-209a-498f-b513-d053a1773cb7 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:57:03.020602 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4b4418e5-209a-498f-b513-d053a1773cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:57:03.050902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:57:03.086628 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 648/1294] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:57:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:57:03.110771 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e2ba1-e50c-460b-84b8-f8da3c9c4693 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:57:03.142385 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:57:03.212797 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e2ba1-e50c-460b-84b8-f8da3c9c4693 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:57:03.248916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047066d-a0cf-407b-a855-0c079f2d754f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:57:03.249572 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047066d-a0cf-407b-a855-0c079f2d754f None admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:03.249854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047066d-a0cf-407b-a855-0c079f2d754f None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:03.250129 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047066d-a0cf-407b-a855-0c079f2d754f None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:03.251091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047066d-a0cf-407b-a855-0c079f2d754f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:57:03.252518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047066d-a0cf-407b-a855-0c079f2d754f None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:57:03.333544 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 647/1295] 10.222.0.83 () {60 vars in 1141 bytes} [Mon Jul 18 14:57:03 2022] POST /identity/v3/services => generated 246 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:57:05.429010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5320183-06fd-4f37-b2e8-268d5323204f None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:05.429626 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5320183-06fd-4f37-b2e8-268d5323204f None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:05.429626 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5320183-06fd-4f37-b2e8-268d5323204f None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:05.431009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 649/1296] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:57:05.436581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac85ea4-2234-4179-9582-8db7ef43ffd8 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:05.437071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac85ea4-2234-4179-9582-8db7ef43ffd8 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:05.437190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac85ea4-2234-4179-9582-8db7ef43ffd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:05.457025 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-6ac85ea4-2234-4179-9582-8db7ef43ffd8 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:57:05.458272 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6ac85ea4-2234-4179-9582-8db7ef43ffd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:57:05.536327 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 648/1297] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:57:05 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:57:05.559860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5320183-06fd-4f37-b2e8-268d5323204f None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:57:05.665037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5320183-06fd-4f37-b2e8-268d5323204f None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:57:05.699438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6965ba-ee48-45f4-ad46-12e61218e9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:57:05.700161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6965ba-ee48-45f4-ad46-12e61218e9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:05.700451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6965ba-ee48-45f4-ad46-12e61218e9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:05.700727 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6965ba-ee48-45f4-ad46-12e61218e9d7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:05.701378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6965ba-ee48-45f4-ad46-12e61218e9d7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:57:05.702796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6965ba-ee48-45f4-ad46-12e61218e9d7 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-8b6965ba-ee48-45f4-ad46-12e61218e9d7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:57:05.714748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:57:05.715985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:57:05.715985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:57:05.715985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:57:05.715985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:57:05.715985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:57:05.715985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:57:05.715985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:57:05.715985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:57:05.715985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:57:05.715985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:57:05.715985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:57:05.715985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:57:05.715985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:57:05.715985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:57:05.715985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:57:05.715985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:57:05.717101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:57:05.717101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:57:05.717101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:57:05.717101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:57:05.717101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:57:05.717101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:57:05.717101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:57:05.717101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:57:05.717101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:57:05.717101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:57:05.717101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:57:05.717101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:57:05.717101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:57:05.717101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:57:05.717101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 650/1298] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Jul 18 14:57:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:57:05.722813 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac85ea4-2234-4179-9582-8db7ef43ffd8 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:57:05.741963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:57:05.827686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac85ea4-2234-4179-9582-8db7ef43ffd8 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:57:05.864358 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5268a2f-b3c3-4c6e-840d-c2e429603957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:57:05.865017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5268a2f-b3c3-4c6e-840d-c2e429603957 None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:05.865298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5268a2f-b3c3-4c6e-840d-c2e429603957 None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:05.865570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5268a2f-b3c3-4c6e-840d-c2e429603957 None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:05.866271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5268a2f-b3c3-4c6e-840d-c2e429603957 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:57:05.866607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5268a2f-b3c3-4c6e-840d-c2e429603957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:57:05.867978 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5268a2f-b3c3-4c6e-840d-c2e429603957 None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:57:05.873113 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 649/1299] 10.222.0.83 () {56 vars in 1116 bytes} [Mon Jul 18 14:57:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:57:05.880047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6965ba-ee48-45f4-ad46-12e61218e9d7 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:57:05.913066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:57:05.980755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6965ba-ee48-45f4-ad46-12e61218e9d7 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:57:06.016592 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d36398-1920-437f-9380-ff480cf7722e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:57:06.017860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d36398-1920-437f-9380-ff480cf7722e None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:06.018152 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d36398-1920-437f-9380-ff480cf7722e None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:06.018426 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d36398-1920-437f-9380-ff480cf7722e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:06.019243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d36398-1920-437f-9380-ff480cf7722e None admin] RBAC: Adding query filter params (interface=public, service_id=ee5a6917d4d5438b874e2d9bb388831a, region_id=RegionOne) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:57:06.019553 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d36398-1920-437f-9380-ff480cf7722e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:57:06.020927 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d36398-1920-437f-9380-ff480cf7722e None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:57:06.025874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 651/1300] 10.222.0.83 () {56 vars in 1251 bytes} [Mon Jul 18 14:57:05 2022] GET /identity/v3/endpoints?service_id=ee5a6917d4d5438b874e2d9bb388831a&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:57:06.031833 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5268a2f-b3c3-4c6e-840d-c2e429603957 None admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:57:06.117642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:57:06.136398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5268a2f-b3c3-4c6e-840d-c2e429603957 None admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:57:06.172842 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff3c18-ac87-448c-a470-79ce4a613aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:57:06.174709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff3c18-ac87-448c-a470-79ce4a613aee None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:06.174834 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff3c18-ac87-448c-a470-79ce4a613aee None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:06.175088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff3c18-ac87-448c-a470-79ce4a613aee None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:06.175769 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff3c18-ac87-448c-a470-79ce4a613aee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:57:06.177148 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff3c18-ac87-448c-a470-79ce4a613aee None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:57:06.185023 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 650/1301] 10.222.0.83 () {56 vars in 1087 bytes} [Mon Jul 18 14:57:06 2022] GET /identity/v3/services => generated 2191 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:57:08.228886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5265d-f903-46f0-8c0a-d1d1f4815491 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:08.229523 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5265d-f903-46f0-8c0a-d1d1f4815491 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:08.229523 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5265d-f903-46f0-8c0a-d1d1f4815491 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:08.230785 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 652/1302] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 14:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:57:08.236034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1a852-1101-4862-80f2-05e9850e9d50 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:08.236473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1a852-1101-4862-80f2-05e9850e9d50 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:08.236627 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1a852-1101-4862-80f2-05e9850e9d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:08.254130 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b2f1a852-1101-4862-80f2-05e9850e9d50 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:57:08.255360 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b2f1a852-1101-4862-80f2-05e9850e9d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:57:08.324357 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 651/1303] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 14:57:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:57:08.348217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5265d-f903-46f0-8c0a-d1d1f4815491 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:57:08.443913 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:57:08.450800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5265d-f903-46f0-8c0a-d1d1f4815491 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:57:08.485692 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1888cb0-3879-406c-a14d-3d559058a133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:57:08.486522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1888cb0-3879-406c-a14d-3d559058a133 None admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:08.486838 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1888cb0-3879-406c-a14d-3d559058a133 None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:08.487123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1888cb0-3879-406c-a14d-3d559058a133 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:08.487773 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1888cb0-3879-406c-a14d-3d559058a133 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:57:08.489218 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1888cb0-3879-406c-a14d-3d559058a133 None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application [None req-e1888cb0-3879-406c-a14d-3d559058a133 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:57:08.502683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:57:08.507149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:57:08.507149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:57:08.507149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:57:08.507149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:57:08.507149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:57:08.507149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:57:08.507149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:57:08.507149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:57:08.507149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:57:08.507149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:57:08.507149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:57:08.507149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:57:08.507149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:57:08.507149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:57:08.507149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:57:08.507149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:57:08.508233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:57:08.508233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application with Lock( Jul 18 14:57:08.508233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:57:08.508233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:57:08.508233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:57:08.508233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:57:08.508233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:57:08.508233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:57:08.508233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:57:08.508233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:57:08.508233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:57:08.508233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:57:08.508233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:57:08.508233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: ERROR keystone.server.flask.application Jul 18 14:57:08.508233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 653/1304] 10.222.0.83 () {56 vars in 1117 bytes} [Mon Jul 18 14:57:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:57:08.510091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1a852-1101-4862-80f2-05e9850e9d50 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:57:08.589259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:57:08.612814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1a852-1101-4862-80f2-05e9850e9d50 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:57:08.648320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbba75d-e7c6-44ed-80e6-e7b8af33307b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:57:08.648989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbba75d-e7c6-44ed-80e6-e7b8af33307b None admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:08.649275 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbba75d-e7c6-44ed-80e6-e7b8af33307b None admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:08.649554 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbba75d-e7c6-44ed-80e6-e7b8af33307b None admin] PATH_INFO: `/v3/services` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:08.650264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbba75d-e7c6-44ed-80e6-e7b8af33307b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:57:08.650591 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbba75d-e7c6-44ed-80e6-e7b8af33307b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:57:08.652905 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbba75d-e7c6-44ed-80e6-e7b8af33307b None admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:57:08.657359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 652/1305] 10.222.0.83 () {56 vars in 1116 bytes} [Mon Jul 18 14:57:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:57:08.664045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1888cb0-3879-406c-a14d-3d559058a133 None admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:57:08.764621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1888cb0-3879-406c-a14d-3d559058a133 None admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:57:08.798051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1ce03-b6de-44cb-9dc4-9d4e3b51bdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:57:08.799447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1ce03-b6de-44cb-9dc4-9d4e3b51bdde None admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:57:08.799602 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1ce03-b6de-44cb-9dc4-9d4e3b51bdde None admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:57:08.799866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1ce03-b6de-44cb-9dc4-9d4e3b51bdde None admin] PATH_INFO: `/v3/endpoints` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:57:08.800599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1ce03-b6de-44cb-9dc4-9d4e3b51bdde None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:57:08.801987 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1ce03-b6de-44cb-9dc4-9d4e3b51bdde None admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:57:08.916184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 654/1306] 10.222.0.83 () {60 vars in 1144 bytes} [Mon Jul 18 14:57:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:57:18.654159 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=30 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:57:18.911392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:23.049743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f35f4-8db2-4c59-b609-fcd04276a0e2 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:23.050951 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f35f4-8db2-4c59-b609-fcd04276a0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:23.051484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f35f4-8db2-4c59-b609-fcd04276a0e2 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:23.053119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 653/1307] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:04:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:04:23.058805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f477e9-41d4-40a5-9fe2-514c7a69ec61 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:23.059291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f477e9-41d4-40a5-9fe2-514c7a69ec61 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:23.059383 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f477e9-41d4-40a5-9fe2-514c7a69ec61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:23.203250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-b4f477e9-41d4-40a5-9fe2-514c7a69ec61 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:04:23.204643 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b4f477e9-41d4-40a5-9fe2-514c7a69ec61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:23.354574 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 655/1308] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:04:23.429222 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64716e6-7efd-4c0c-9a35-f38787450624 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:23.430240 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64716e6-7efd-4c0c-9a35-f38787450624 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:23.431053 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64716e6-7efd-4c0c-9a35-f38787450624 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:23.433962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 654/1309] 10.222.0.83 () {54 vars in 965 bytes} [Mon Jul 18 15:04:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:04:23.443941 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35520f-f8f4-483d-81bd-03908f1b0b67 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:23.444258 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35520f-f8f4-483d-81bd-03908f1b0b67 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:23.444677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35520f-f8f4-483d-81bd-03908f1b0b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:23.557075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-2b35520f-f8f4-483d-81bd-03908f1b0b67 None None] MFA Rules not processed for user `fd86979e0f0d4a4e9921ee5704901f74`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:04:23.558322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-2b35520f-f8f4-483d-81bd-03908f1b0b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:23.581897 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:23.669430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 656/1310] 10.222.0.83 () {58 vars in 1062 bytes} [Mon Jul 18 15:04:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:04:23.679324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64716e6-7efd-4c0c-9a35-f38787450624 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:04:23.680989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b64716e6-7efd-4c0c-9a35-f38787450624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:23.819943 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64716e6-7efd-4c0c-9a35-f38787450624 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:04:23.920626 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0da73-ee95-4785-bb2f-e6a8d265da08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:04:23.921466 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0da73-ee95-4785-bb2f-e6a8d265da08 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:23.921620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0da73-ee95-4785-bb2f-e6a8d265da08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:23.922049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0da73-ee95-4785-bb2f-e6a8d265da08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:23.923721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e6b0da73-ee95-4785-bb2f-e6a8d265da08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:23.992122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:24.067276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0da73-ee95-4785-bb2f-e6a8d265da08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:04:24.068853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0da73-ee95-4785-bb2f-e6a8d265da08 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:04:24.097147 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 655/1311] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:04:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:04:26.501064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863e909-f166-488f-b728-a6ff8b216972 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:26.501754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863e909-f166-488f-b728-a6ff8b216972 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:26.501754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863e909-f166-488f-b728-a6ff8b216972 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:26.503268 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 657/1312] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:04:26.508604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0792d94-5bb5-4084-a852-31c24a142924 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:26.509137 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0792d94-5bb5-4084-a852-31c24a142924 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:26.509236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0792d94-5bb5-4084-a852-31c24a142924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:26.599805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-c0792d94-5bb5-4084-a852-31c24a142924 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:04:26.601430 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c0792d94-5bb5-4084-a852-31c24a142924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:26.715549 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 656/1313] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:04:26.756470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863e909-f166-488f-b728-a6ff8b216972 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:04:26.757960 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b863e909-f166-488f-b728-a6ff8b216972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:26.875376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863e909-f166-488f-b728-a6ff8b216972 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:04:26.957283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:26.959927 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295c29e-09be-49d6-80d8-98f64abfed61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:04:26.960580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295c29e-09be-49d6-80d8-98f64abfed61 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:26.960863 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295c29e-09be-49d6-80d8-98f64abfed61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:26.961137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295c29e-09be-49d6-80d8-98f64abfed61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:26.962841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-2295c29e-09be-49d6-80d8-98f64abfed61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:27.079937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295c29e-09be-49d6-80d8-98f64abfed61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:04:27.081420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295c29e-09be-49d6-80d8-98f64abfed61 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:04:27.101509 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 658/1314] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:04:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:04:29.764920 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897ebd0-7bb8-4edb-b37c-c64d5b0b3df7 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:29.765536 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897ebd0-7bb8-4edb-b37c-c64d5b0b3df7 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:29.765643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897ebd0-7bb8-4edb-b37c-c64d5b0b3df7 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:29.767233 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 657/1315] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:04:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:04:29.773294 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b5aa6-c001-4b08-be9c-2fccd11e3052 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:29.773907 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b5aa6-c001-4b08-be9c-2fccd11e3052 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:29.773907 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b5aa6-c001-4b08-be9c-2fccd11e3052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:29.864824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-157b5aa6-c001-4b08-be9c-2fccd11e3052 None None] MFA Rules not processed for user `b7bfc576d3154257a676f91304b0d7b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:04:29.866142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-157b5aa6-c001-4b08-be9c-2fccd11e3052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:29.937345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:29.999946 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 659/1316] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:04:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:04:30.047775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897ebd0-7bb8-4edb-b37c-c64d5b0b3df7 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:04:30.049345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c897ebd0-7bb8-4edb-b37c-c64d5b0b3df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:30.059509 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:30.160419 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897ebd0-7bb8-4edb-b37c-c64d5b0b3df7 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:04:30.246886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1410c78-ae7e-4287-9da6-cb4d8c82e868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:04:30.247652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1410c78-ae7e-4287-9da6-cb4d8c82e868 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:30.247934 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1410c78-ae7e-4287-9da6-cb4d8c82e868 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:30.248216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1410c78-ae7e-4287-9da6-cb4d8c82e868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:30.250587 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a1410c78-ae7e-4287-9da6-cb4d8c82e868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:30.328104 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:30.386102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1410c78-ae7e-4287-9da6-cb4d8c82e868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:04:30.387856 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1410c78-ae7e-4287-9da6-cb4d8c82e868 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:04:30.408597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 658/1317] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:04:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:04:33.375449 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691f53a-04b2-4612-8fea-c7d38f58dae4 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:33.376022 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691f53a-04b2-4612-8fea-c7d38f58dae4 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:33.376022 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691f53a-04b2-4612-8fea-c7d38f58dae4 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:33.377621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 660/1318] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:04:33.383187 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d0a5f-d420-4827-82e0-b53e7fbae99a None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:33.383618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d0a5f-d420-4827-82e0-b53e7fbae99a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:33.383795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d0a5f-d420-4827-82e0-b53e7fbae99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:33.452260 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-657d0a5f-d420-4827-82e0-b53e7fbae99a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:04:33.453535 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-657d0a5f-d420-4827-82e0-b53e7fbae99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:33.552900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 659/1319] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:04:33.609395 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ff8c1-a70c-4ceb-9fcc-0f6f1c404415 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:33.610015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ff8c1-a70c-4ceb-9fcc-0f6f1c404415 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:33.610302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ff8c1-a70c-4ceb-9fcc-0f6f1c404415 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:33.611781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 661/1320] 10.222.0.83 () {54 vars in 965 bytes} [Mon Jul 18 15:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:04:33.618091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4246239-576f-4ae6-93af-b309e17e1b4f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:33.618381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4246239-576f-4ae6-93af-b309e17e1b4f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:33.618786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4246239-576f-4ae6-93af-b309e17e1b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:33.640275 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:33.694698 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-c4246239-576f-4ae6-93af-b309e17e1b4f None None] MFA Rules not processed for user `fd86979e0f0d4a4e9921ee5704901f74`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:04:33.695964 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c4246239-576f-4ae6-93af-b309e17e1b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:33.774030 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 660/1321] 10.222.0.83 () {58 vars in 1062 bytes} [Mon Jul 18 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:04:33.783729 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ff8c1-a70c-4ceb-9fcc-0f6f1c404415 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:04:33.785262 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a89ff8c1-a70c-4ceb-9fcc-0f6f1c404415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:33.894491 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ff8c1-a70c-4ceb-9fcc-0f6f1c404415 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:04:33.977605 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20bd262-76a3-4edc-a8ff-c3fd499ec6bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:04:33.978264 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bd262-76a3-4edc-a8ff-c3fd499ec6bb service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:33.978572 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bd262-76a3-4edc-a8ff-c3fd499ec6bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:33.978858 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bd262-76a3-4edc-a8ff-c3fd499ec6bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:33.980579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b20bd262-76a3-4edc-a8ff-c3fd499ec6bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:33.989218 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:34.098516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20bd262-76a3-4edc-a8ff-c3fd499ec6bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:04:34.100022 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20bd262-76a3-4edc-a8ff-c3fd499ec6bb service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:04:34.120159 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 662/1322] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:04:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:04:37.699392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de42863-7837-4513-a968-457094fcc8a9 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:37.700152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de42863-7837-4513-a968-457094fcc8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:37.700152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de42863-7837-4513-a968-457094fcc8a9 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:37.701711 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 661/1323] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:04:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:04:37.707437 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308e7c3-b879-4984-af97-2e7e835e5ba5 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:37.708774 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308e7c3-b879-4984-af97-2e7e835e5ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:37.708774 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308e7c3-b879-4984-af97-2e7e835e5ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:37.779222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-e308e7c3-b879-4984-af97-2e7e835e5ba5 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:04:37.780461 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e308e7c3-b879-4984-af97-2e7e835e5ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:37.839230 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:37.881176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 663/1324] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:04:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:04:37.919784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de42863-7837-4513-a968-457094fcc8a9 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:04:37.921339 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-8de42863-7837-4513-a968-457094fcc8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:38.044411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de42863-7837-4513-a968-457094fcc8a9 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:04:38.058693 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:38.132430 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b71c2-7b19-42b6-ae5c-347977d67a08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:04:38.133085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b71c2-7b19-42b6-ae5c-347977d67a08 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:38.133369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b71c2-7b19-42b6-ae5c-347977d67a08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:38.133646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b71c2-7b19-42b6-ae5c-347977d67a08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:38.135578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-595b71c2-7b19-42b6-ae5c-347977d67a08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:38.262169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b71c2-7b19-42b6-ae5c-347977d67a08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:04:38.263696 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b71c2-7b19-42b6-ae5c-347977d67a08 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:04:38.284667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 662/1325] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:04:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:04:40.436695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7fdd9-a8ff-4455-8cc8-9b1cbae22f04 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:40.437278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7fdd9-a8ff-4455-8cc8-9b1cbae22f04 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:40.437278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7fdd9-a8ff-4455-8cc8-9b1cbae22f04 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:40.438765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 664/1326] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:04:40.444725 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80879da-8ea4-40f6-a57e-64c0975b4d52 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:40.445314 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80879da-8ea4-40f6-a57e-64c0975b4d52 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:40.445314 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80879da-8ea4-40f6-a57e-64c0975b4d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:40.495967 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:40.516628 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-c80879da-8ea4-40f6-a57e-64c0975b4d52 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:04:40.517937 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c80879da-8ea4-40f6-a57e-64c0975b4d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:40.622908 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 663/1327] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:04:40.660982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7fdd9-a8ff-4455-8cc8-9b1cbae22f04 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:04:40.662687 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-10a7fdd9-a8ff-4455-8cc8-9b1cbae22f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:40.772994 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7fdd9-a8ff-4455-8cc8-9b1cbae22f04 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:04:40.849966 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a120846-0531-4340-95f4-b1ce0f5b739a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:04:40.850642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a120846-0531-4340-95f4-b1ce0f5b739a service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:40.850932 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a120846-0531-4340-95f4-b1ce0f5b739a service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:40.851210 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a120846-0531-4340-95f4-b1ce0f5b739a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:40.853133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-0a120846-0531-4340-95f4-b1ce0f5b739a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:40.871489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:40.969855 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a120846-0531-4340-95f4-b1ce0f5b739a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:04:40.971363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a120846-0531-4340-95f4-b1ce0f5b739a service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:04:40.993313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 665/1328] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:04:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:04:43.142583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ba837-2c28-4f66-ad8b-53e522e91691 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:43.143145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ba837-2c28-4f66-ad8b-53e522e91691 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:43.143145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ba837-2c28-4f66-ad8b-53e522e91691 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:43.144573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 664/1329] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:04:43.150018 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12d135-a62d-4233-9a58-9ccac1cec2ce None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:43.150478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12d135-a62d-4233-9a58-9ccac1cec2ce None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:43.150766 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12d135-a62d-4233-9a58-9ccac1cec2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:43.219575 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-4d12d135-a62d-4233-9a58-9ccac1cec2ce None None] MFA Rules not processed for user `b7bfc576d3154257a676f91304b0d7b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:04:43.220819 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4d12d135-a62d-4233-9a58-9ccac1cec2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:43.302218 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:43.338064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 666/1330] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:04:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:04:43.383353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ba837-2c28-4f66-ad8b-53e522e91691 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:04:43.384927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-595ba837-2c28-4f66-ad8b-53e522e91691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:43.510770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ba837-2c28-4f66-ad8b-53e522e91691 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:04:43.534377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:43.594196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c8bc1-7855-4892-bcef-efd1a199478b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:04:43.594686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c8bc1-7855-4892-bcef-efd1a199478b service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:43.594686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c8bc1-7855-4892-bcef-efd1a199478b service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:43.594944 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c8bc1-7855-4892-bcef-efd1a199478b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:43.596672 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2e1c8bc1-7855-4892-bcef-efd1a199478b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:43.715660 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c8bc1-7855-4892-bcef-efd1a199478b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:04:43.717109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c8bc1-7855-4892-bcef-efd1a199478b service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:04:43.737382 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 665/1331] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:04:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:04:46.299352 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a808a-8359-4d92-a836-33fa5f180446 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:46.299980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a808a-8359-4d92-a836-33fa5f180446 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:46.299980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a808a-8359-4d92-a836-33fa5f180446 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:46.301282 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 667/1332] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:04:46.306620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb7b63-adb5-48e2-9fdb-c4de9f406fe2 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:46.307051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb7b63-adb5-48e2-9fdb-c4de9f406fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:46.307194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb7b63-adb5-48e2-9fdb-c4de9f406fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:46.374496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:46.377316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-70fb7b63-adb5-48e2-9fdb-c4de9f406fe2 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:04:46.378601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-70fb7b63-adb5-48e2-9fdb-c4de9f406fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:46.488463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 666/1333] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:04:46.526976 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a808a-8359-4d92-a836-33fa5f180446 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:04:46.529305 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-847a808a-8359-4d92-a836-33fa5f180446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:46.640035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a808a-8359-4d92-a836-33fa5f180446 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:04:46.718046 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d43535-a59f-469e-9124-78c015a13df6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:04:46.718738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d43535-a59f-469e-9124-78c015a13df6 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:46.719022 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d43535-a59f-469e-9124-78c015a13df6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:46.719298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d43535-a59f-469e-9124-78c015a13df6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:46.721142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-62d43535-a59f-469e-9124-78c015a13df6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:46.750000 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:46.849863 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d43535-a59f-469e-9124-78c015a13df6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:04:46.851426 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d43535-a59f-469e-9124-78c015a13df6 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:04:46.874047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 668/1334] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:04:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:04:49.086911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc3f38-c5f9-4e06-b389-f9746c1f03a1 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:49.087480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc3f38-c5f9-4e06-b389-f9746c1f03a1 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:49.087480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc3f38-c5f9-4e06-b389-f9746c1f03a1 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:49.088912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 667/1335] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:04:49.095505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288f063-602b-4380-87b7-dff8951c6115 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:49.096055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288f063-602b-4380-87b7-dff8951c6115 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:49.096468 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288f063-602b-4380-87b7-dff8951c6115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:49.175926 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-a288f063-602b-4380-87b7-dff8951c6115 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:04:49.176803 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a288f063-602b-4380-87b7-dff8951c6115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:49.261254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:49.285620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 669/1336] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:04:49.328088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc3f38-c5f9-4e06-b389-f9746c1f03a1 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:04:49.329645 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-42dc3f38-c5f9-4e06-b389-f9746c1f03a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:49.464623 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc3f38-c5f9-4e06-b389-f9746c1f03a1 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:04:49.495416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:49.546051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f80559-8ea7-4545-a2af-0aa249f80758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:04:49.546742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f80559-8ea7-4545-a2af-0aa249f80758 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:49.547029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f80559-8ea7-4545-a2af-0aa249f80758 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:49.547307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f80559-8ea7-4545-a2af-0aa249f80758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:49.548996 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-95f80559-8ea7-4545-a2af-0aa249f80758 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:49.669170 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f80559-8ea7-4545-a2af-0aa249f80758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:04:49.670744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f80559-8ea7-4545-a2af-0aa249f80758 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:04:49.690940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 668/1337] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:04:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:04:52.396488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdf103-0b53-4efb-b785-26e0a84552ba None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:52.397148 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdf103-0b53-4efb-b785-26e0a84552ba None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:52.397260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdf103-0b53-4efb-b785-26e0a84552ba None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:52.398670 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 670/1338] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:04:52.404740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa312590-6bb1-495d-b6b4-feb521e17745 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:52.405318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa312590-6bb1-495d-b6b4-feb521e17745 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:52.405318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa312590-6bb1-495d-b6b4-feb521e17745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:52.474212 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-aa312590-6bb1-495d-b6b4-feb521e17745 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:04:52.475544 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-aa312590-6bb1-495d-b6b4-feb521e17745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:52.485407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:52.575342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 669/1339] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:04:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:04:52.612209 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdf103-0b53-4efb-b785-26e0a84552ba None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:04:52.613771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a9bdf103-0b53-4efb-b785-26e0a84552ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:52.722826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdf103-0b53-4efb-b785-26e0a84552ba None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:04:52.801255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e4c2f-0f13-42bc-9e30-11c0f4592e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:04:52.801912 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e4c2f-0f13-42bc-9e30-11c0f4592e07 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:52.802197 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e4c2f-0f13-42bc-9e30-11c0f4592e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:52.802689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e4c2f-0f13-42bc-9e30-11c0f4592e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:52.805372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-420e4c2f-0f13-42bc-9e30-11c0f4592e07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:52.844488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:52.922432 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e4c2f-0f13-42bc-9e30-11c0f4592e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:04:52.924056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e4c2f-0f13-42bc-9e30-11c0f4592e07 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:04:52.943733 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 671/1340] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:04:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:04:57.333207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039aef9e-e7b4-41e2-99f7-aa79be8486c7 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:57.333754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039aef9e-e7b4-41e2-99f7-aa79be8486c7 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:57.333754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039aef9e-e7b4-41e2-99f7-aa79be8486c7 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:57.335112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 670/1341] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:04:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:04:57.341616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8f7aa-aec0-4f8b-bf7e-5d1fc8c634b7 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:57.342169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8f7aa-aec0-4f8b-bf7e-5d1fc8c634b7 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:57.342169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8f7aa-aec0-4f8b-bf7e-5d1fc8c634b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:57.425016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-bce8f7aa-aec0-4f8b-bf7e-5d1fc8c634b7 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:04:57.426268 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-bce8f7aa-aec0-4f8b-bf7e-5d1fc8c634b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:57.538220 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:57.551168 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 672/1342] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:04:57.591504 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039aef9e-e7b4-41e2-99f7-aa79be8486c7 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:04:57.593058 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-039aef9e-e7b4-41e2-99f7-aa79be8486c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:57.706112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039aef9e-e7b4-41e2-99f7-aa79be8486c7 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:04:57.747512 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:04:57.784189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ca26f-1fdd-46b4-9bd0-45d01f3cad99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:04:57.784849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ca26f-1fdd-46b4-9bd0-45d01f3cad99 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:04:57.785407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ca26f-1fdd-46b4-9bd0-45d01f3cad99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:04:57.785804 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ca26f-1fdd-46b4-9bd0-45d01f3cad99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:04:57.788022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ac4ca26f-1fdd-46b4-9bd0-45d01f3cad99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:04:57.903969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ca26f-1fdd-46b4-9bd0-45d01f3cad99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:04:57.905334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ca26f-1fdd-46b4-9bd0-45d01f3cad99 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:04:57.926234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 671/1343] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:04:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:05:00.924580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae23ea-3d6d-4fea-b702-9e0c9673cda2 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:00.925236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae23ea-3d6d-4fea-b702-9e0c9673cda2 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:00.925236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae23ea-3d6d-4fea-b702-9e0c9673cda2 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:00.926750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 673/1344] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:00.932831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af35b78-9d6f-47ae-a947-8b8783d671ce None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:00.933798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af35b78-9d6f-47ae-a947-8b8783d671ce None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:00.934087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af35b78-9d6f-47ae-a947-8b8783d671ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:01.008617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-0af35b78-9d6f-47ae-a947-8b8783d671ce None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:01.009835 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0af35b78-9d6f-47ae-a947-8b8783d671ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:01.029827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:01.114169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 672/1345] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:05:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:01.157792 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae23ea-3d6d-4fea-b702-9e0c9673cda2 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:01.160069 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-8fae23ea-3d6d-4fea-b702-9e0c9673cda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:01.269312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae23ea-3d6d-4fea-b702-9e0c9673cda2 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:01.358576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f17b25-874e-48c2-ab2a-0540325ac29a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:01.359236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f17b25-874e-48c2-ab2a-0540325ac29a service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:01.359523 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f17b25-874e-48c2-ab2a-0540325ac29a service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:01.359804 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f17b25-874e-48c2-ab2a-0540325ac29a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:01.361482 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-45f17b25-874e-48c2-ab2a-0540325ac29a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:01.409160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:01.481198 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f17b25-874e-48c2-ab2a-0540325ac29a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:01.482800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f17b25-874e-48c2-ab2a-0540325ac29a service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:01.503975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 674/1346] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:05:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:05:05.896258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b4100-c66a-4c71-b7fb-ad1b4ffdc018 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:05.896853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b4100-c66a-4c71-b7fb-ad1b4ffdc018 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:05.896853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b4100-c66a-4c71-b7fb-ad1b4ffdc018 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:05.898260 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 673/1347] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:05:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:05.905205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c131eb6-13bc-483c-9879-7b641e81a145 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:05.905692 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c131eb6-13bc-483c-9879-7b641e81a145 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:05.905790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c131eb6-13bc-483c-9879-7b641e81a145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:06.012691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-8c131eb6-13bc-483c-9879-7b641e81a145 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:06.013973 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-8c131eb6-13bc-483c-9879-7b641e81a145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:06.123170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:06.125654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 675/1348] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:06.166492 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b4100-c66a-4c71-b7fb-ad1b4ffdc018 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:06.168702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-269b4100-c66a-4c71-b7fb-ad1b4ffdc018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:06.285560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b4100-c66a-4c71-b7fb-ad1b4ffdc018 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:06.339926 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:06.371419 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e4a2a-5355-4029-a330-0e5205d51a56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:06.372105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e4a2a-5355-4029-a330-0e5205d51a56 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:06.372394 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e4a2a-5355-4029-a330-0e5205d51a56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:06.372677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e4a2a-5355-4029-a330-0e5205d51a56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:06.374519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-fa8e4a2a-5355-4029-a330-0e5205d51a56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:06.503153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e4a2a-5355-4029-a330-0e5205d51a56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:06.504589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e4a2a-5355-4029-a330-0e5205d51a56 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:06.527348 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 674/1349] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:05:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:05:12.027422 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0a759-f0c5-4ec6-b0b8-39f16808a046 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:12.027981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0a759-f0c5-4ec6-b0b8-39f16808a046 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:12.028238 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0a759-f0c5-4ec6-b0b8-39f16808a046 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:12.029744 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 676/1350] 10.222.0.83 () {54 vars in 921 bytes} [Mon Jul 18 15:05:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:12.038412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b48fb5-25f1-403f-ac5d-f64e663904ee None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:12.040249 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b48fb5-25f1-403f-ac5d-f64e663904ee None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:12.040969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b48fb5-25f1-403f-ac5d-f64e663904ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:12.145679 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-e6b48fb5-25f1-403f-ac5d-f64e663904ee None None] MFA Rules not processed for user `76cc0ae48b5a49c6a7a1c11a08c389e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:12.147141 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e6b48fb5-25f1-403f-ac5d-f64e663904ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:12.173855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:12.287580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 675/1351] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Jul 18 15:05:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:16.928295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20600394-dbb6-4469-9088-028ee91b4224 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:16.929189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20600394-dbb6-4469-9088-028ee91b4224 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:16.929672 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20600394-dbb6-4469-9088-028ee91b4224 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:16.932128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 677/1352] 10.222.0.83 () {54 vars in 921 bytes} [Mon Jul 18 15:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:16.938179 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc931b-bf2c-4d2c-ba11-fc524ea9cfb6 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:16.938715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc931b-bf2c-4d2c-ba11-fc524ea9cfb6 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:16.939037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc931b-bf2c-4d2c-ba11-fc524ea9cfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:17.033212 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-fdbc931b-bf2c-4d2c-ba11-fc524ea9cfb6 None None] MFA Rules not processed for user `76cc0ae48b5a49c6a7a1c11a08c389e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:17.034454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-fdbc931b-bf2c-4d2c-ba11-fc524ea9cfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:17.113926 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:17.155321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 676/1353] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Jul 18 15:05:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:05:18.567796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdf833-8a00-4616-a908-7807b5e4dfca None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:18.568370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdf833-8a00-4616-a908-7807b5e4dfca None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:18.568370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdf833-8a00-4616-a908-7807b5e4dfca None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:18.569729 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 678/1354] 10.222.0.83 () {54 vars in 921 bytes} [Mon Jul 18 15:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:18.575657 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d32725-9f9d-46c7-ada2-b1bb0fe14a1e None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:18.576145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d32725-9f9d-46c7-ada2-b1bb0fe14a1e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:18.576346 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d32725-9f9d-46c7-ada2-b1bb0fe14a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:18.658435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-65d32725-9f9d-46c7-ada2-b1bb0fe14a1e None None] MFA Rules not processed for user `76cc0ae48b5a49c6a7a1c11a08c389e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:18.659717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-65d32725-9f9d-46c7-ada2-b1bb0fe14a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:18.777951 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 677/1355] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Jul 18 15:05:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:22.387754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6a242-4941-4423-ac69-0a2c333a3dbc None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:22.388351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6a242-4941-4423-ac69-0a2c333a3dbc None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:22.388351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6a242-4941-4423-ac69-0a2c333a3dbc None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:22.389750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 679/1356] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:22.395462 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec2cce-2295-4eed-bd59-d5cbd38c7353 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:22.395941 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec2cce-2295-4eed-bd59-d5cbd38c7353 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:22.396053 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec2cce-2295-4eed-bd59-d5cbd38c7353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:22.433397 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:22.473501 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-1fec2cce-2295-4eed-bd59-d5cbd38c7353 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:22.474329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-1fec2cce-2295-4eed-bd59-d5cbd38c7353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:22.633680 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 678/1357] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:22.697941 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbed4e-de05-42f9-bbde-2adabc4c42a3 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:22.698566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbed4e-de05-42f9-bbde-2adabc4c42a3 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:22.699095 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbed4e-de05-42f9-bbde-2adabc4c42a3 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:22.701334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 680/1358] 10.222.0.83 () {54 vars in 962 bytes} [Mon Jul 18 15:05:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:22.713798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345d942-1e6e-4280-b9c5-24451d1a3580 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:22.752118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345d942-1e6e-4280-b9c5-24451d1a3580 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:22.752118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345d942-1e6e-4280-b9c5-24451d1a3580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:22.855831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-9345d942-1e6e-4280-b9c5-24451d1a3580 None None] MFA Rules not processed for user `7e1355d18fb049468651b3675c217d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:22.858035 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9345d942-1e6e-4280-b9c5-24451d1a3580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:22.929320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:23.033309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 679/1359] 10.222.0.83 () {58 vars in 1059 bytes} [Mon Jul 18 15:05:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:05:23.034126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbed4e-de05-42f9-bbde-2adabc4c42a3 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:23.035741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-2bfbed4e-de05-42f9-bbde-2adabc4c42a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:23.243301 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbed4e-de05-42f9-bbde-2adabc4c42a3 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:23.327670 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:23.371965 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7830a-ba46-49fd-8cc8-a1a9b95c0c3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:23.375262 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7830a-ba46-49fd-8cc8-a1a9b95c0c3a service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:23.375262 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7830a-ba46-49fd-8cc8-a1a9b95c0c3a service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:23.375262 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7830a-ba46-49fd-8cc8-a1a9b95c0c3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:23.375262 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-80a7830a-ba46-49fd-8cc8-a1a9b95c0c3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:23.480671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7830a-ba46-49fd-8cc8-a1a9b95c0c3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:23.482597 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7830a-ba46-49fd-8cc8-a1a9b95c0c3a service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:23.505982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 681/1360] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:05:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:05:25.970980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938db580-558a-45ab-bbf5-0653b6a5f1e4 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:25.971588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938db580-558a-45ab-bbf5-0653b6a5f1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:25.971588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938db580-558a-45ab-bbf5-0653b6a5f1e4 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:25.973183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 680/1361] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:25.979464 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343bfac-9652-4196-8bd7-5f46d4155005 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:25.979999 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343bfac-9652-4196-8bd7-5f46d4155005 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:25.979999 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343bfac-9652-4196-8bd7-5f46d4155005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:26.052404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-9343bfac-9652-4196-8bd7-5f46d4155005 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:26.053628 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-9343bfac-9652-4196-8bd7-5f46d4155005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:26.096214 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:26.170610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 682/1362] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:05:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:26.219816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938db580-558a-45ab-bbf5-0653b6a5f1e4 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:26.222085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-938db580-558a-45ab-bbf5-0653b6a5f1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:26.419663 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938db580-558a-45ab-bbf5-0653b6a5f1e4 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:26.429257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:26.558018 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef9189-862c-4615-9848-81e72baa205d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:26.558747 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef9189-862c-4615-9848-81e72baa205d service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:26.559020 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef9189-862c-4615-9848-81e72baa205d service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:26.559404 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef9189-862c-4615-9848-81e72baa205d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:26.560972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-fcef9189-862c-4615-9848-81e72baa205d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:26.671981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef9189-862c-4615-9848-81e72baa205d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:26.673621 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef9189-862c-4615-9848-81e72baa205d service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:26.694378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 681/1363] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:05:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:05:29.228690 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ade1d-0226-477e-a041-dbcefc35a32e None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:29.229266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ade1d-0226-477e-a041-dbcefc35a32e None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:29.229266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ade1d-0226-477e-a041-dbcefc35a32e None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:29.230550 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 683/1364] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:05:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:29.236351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca52bef-1fb1-4e41-a9f2-64e9413f4ce7 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:29.236972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca52bef-1fb1-4e41-a9f2-64e9413f4ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:29.236972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca52bef-1fb1-4e41-a9f2-64e9413f4ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:29.253560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:29.333321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-7ca52bef-1fb1-4e41-a9f2-64e9413f4ce7 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:29.335603 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7ca52bef-1fb1-4e41-a9f2-64e9413f4ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:29.468875 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 682/1365] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:29.522721 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad957164-9bee-4b44-a3eb-637e512bae34 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:29.523228 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad957164-9bee-4b44-a3eb-637e512bae34 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:29.523900 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad957164-9bee-4b44-a3eb-637e512bae34 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:29.525623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 684/1366] 10.222.0.83 () {54 vars in 962 bytes} [Mon Jul 18 15:05:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:29.531968 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b85bf4-cbae-41b1-b3bf-100a5beb72e6 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:29.532281 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b85bf4-cbae-41b1-b3bf-100a5beb72e6 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:29.532548 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b85bf4-cbae-41b1-b3bf-100a5beb72e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:29.627126 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d6b85bf4-cbae-41b1-b3bf-100a5beb72e6 None None] MFA Rules not processed for user `7e1355d18fb049468651b3675c217d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:29.628102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d6b85bf4-cbae-41b1-b3bf-100a5beb72e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:29.669543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:29.767504 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 683/1367] 10.222.0.83 () {58 vars in 1059 bytes} [Mon Jul 18 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:29.776300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad957164-9bee-4b44-a3eb-637e512bae34 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:29.778003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ad957164-9bee-4b44-a3eb-637e512bae34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:29.964188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad957164-9bee-4b44-a3eb-637e512bae34 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:29.995978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:30.111112 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb3904-8a1a-40f4-917a-0bcb47a3e561 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:30.111869 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb3904-8a1a-40f4-917a-0bcb47a3e561 service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:30.112270 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb3904-8a1a-40f4-917a-0bcb47a3e561 service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:30.112666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb3904-8a1a-40f4-917a-0bcb47a3e561 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:30.114563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a5bb3904-8a1a-40f4-917a-0bcb47a3e561 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:30.221051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb3904-8a1a-40f4-917a-0bcb47a3e561 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:30.222928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb3904-8a1a-40f4-917a-0bcb47a3e561 service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:30.243797 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 685/1368] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:05:31.416117 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86e8ec-0933-4347-839a-afa09aa025b4 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:31.416703 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86e8ec-0933-4347-839a-afa09aa025b4 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:31.416703 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86e8ec-0933-4347-839a-afa09aa025b4 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:31.418305 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 684/1369] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:05:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:31.425858 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabddfff-646c-402f-9ce8-3600febe289f None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:31.426410 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabddfff-646c-402f-9ce8-3600febe289f None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:31.426895 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabddfff-646c-402f-9ce8-3600febe289f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:31.462894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:31.502693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-dabddfff-646c-402f-9ce8-3600febe289f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:31.503967 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-dabddfff-646c-402f-9ce8-3600febe289f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:31.646810 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 686/1370] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:32.042199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fe5dd-3618-4623-9b7c-ceb9b3428bf2 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:32.042494 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fe5dd-3618-4623-9b7c-ceb9b3428bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:32.043095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fe5dd-3618-4623-9b7c-ceb9b3428bf2 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:32.044250 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 685/1371] 10.222.0.83 () {54 vars in 964 bytes} [Mon Jul 18 15:05:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:32.050221 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383df04e-84e2-4c1c-adaa-ce20c457f496 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:32.050678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383df04e-84e2-4c1c-adaa-ce20c457f496 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:32.050998 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383df04e-84e2-4c1c-adaa-ce20c457f496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:32.164515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-383df04e-84e2-4c1c-adaa-ce20c457f496 None None] MFA Rules not processed for user `9218503bc8a647d68ffab6945bae456a`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:32.166060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-383df04e-84e2-4c1c-adaa-ce20c457f496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:32.225018 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:32.265360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 687/1372] 10.222.0.83 () {58 vars in 1061 bytes} [Mon Jul 18 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:32.273902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fe5dd-3618-4623-9b7c-ceb9b3428bf2 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:32.276187 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-060fe5dd-3618-4623-9b7c-ceb9b3428bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:32.417241 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fe5dd-3618-4623-9b7c-ceb9b3428bf2 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:32.469689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:32.513633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8745d-c357-4878-9ab0-e42dc255e25e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:32.514319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8745d-c357-4878-9ab0-e42dc255e25e service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:32.514647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8745d-c357-4878-9ab0-e42dc255e25e service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:32.514936 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8745d-c357-4878-9ab0-e42dc255e25e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:32.516780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-97b8745d-c357-4878-9ab0-e42dc255e25e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:32.646704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b4ed0-bb21-4b86-b020-8a795f9ca787 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:32.647279 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b4ed0-bb21-4b86-b020-8a795f9ca787 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:32.647380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b4ed0-bb21-4b86-b020-8a795f9ca787 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:32.649101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 688/1373] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:05:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 15:05:32.655305 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebf37a-72c7-4f74-8a13-f2b9e85881c0 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:32.655738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebf37a-72c7-4f74-8a13-f2b9e85881c0 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:32.656132 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebf37a-72c7-4f74-8a13-f2b9e85881c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:32.663777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8745d-c357-4878-9ab0-e42dc255e25e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:32.665918 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8745d-c357-4878-9ab0-e42dc255e25e service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:32.686283 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 686/1374] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:05:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:05:32.742557 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-c9ebf37a-72c7-4f74-8a13-f2b9e85881c0 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:32.745261 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c9ebf37a-72c7-4f74-8a13-f2b9e85881c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:32.893583 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 689/1375] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:32.949243 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8745d-c357-4878-9ab0-e42dc255e25e service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:32.952137 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-97b8745d-c357-4878-9ab0-e42dc255e25e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:33.046888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:33.090958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8745d-c357-4878-9ab0-e42dc255e25e service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:33.216946 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17641521-40bc-4654-a2c7-03110e8b25d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:33.217614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17641521-40bc-4654-a2c7-03110e8b25d5 service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:33.217894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17641521-40bc-4654-a2c7-03110e8b25d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:33.218171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17641521-40bc-4654-a2c7-03110e8b25d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:33.219924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-17641521-40bc-4654-a2c7-03110e8b25d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:33.310808 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:33.321619 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17641521-40bc-4654-a2c7-03110e8b25d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:33.327752 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17641521-40bc-4654-a2c7-03110e8b25d5 service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:33.344583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 687/1376] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:05:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:05:35.717064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5157b-72c7-4084-bcfa-62c0164ccd6a None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:35.717659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5157b-72c7-4084-bcfa-62c0164ccd6a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:35.717659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5157b-72c7-4084-bcfa-62c0164ccd6a None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:35.719087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 690/1377] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:35.724743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64abd5e-2947-47eb-adab-e19a80b51584 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:35.725218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64abd5e-2947-47eb-adab-e19a80b51584 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:35.725319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64abd5e-2947-47eb-adab-e19a80b51584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:35.793966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-c64abd5e-2947-47eb-adab-e19a80b51584 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:35.795253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c64abd5e-2947-47eb-adab-e19a80b51584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:35.894902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 688/1378] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:35.947748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5157b-72c7-4084-bcfa-62c0164ccd6a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:35.949208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-81e5157b-72c7-4084-bcfa-62c0164ccd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:36.005870 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:36.131345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5157b-72c7-4084-bcfa-62c0164ccd6a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:36.262785 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee56182-1711-4f41-b860-7622f3f1f2d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:36.263480 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee56182-1711-4f41-b860-7622f3f1f2d2 service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:36.263808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee56182-1711-4f41-b860-7622f3f1f2d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:36.264098 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee56182-1711-4f41-b860-7622f3f1f2d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:36.266972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-aee56182-1711-4f41-b860-7622f3f1f2d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:36.282753 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:36.368091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee56182-1711-4f41-b860-7622f3f1f2d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:36.369769 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee56182-1711-4f41-b860-7622f3f1f2d2 service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:36.391988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 691/1379] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:05:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:05:38.706652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efe87a-97d5-4fbc-8397-094a2e19e62c None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:38.707258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efe87a-97d5-4fbc-8397-094a2e19e62c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:38.707258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efe87a-97d5-4fbc-8397-094a2e19e62c None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:38.708762 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 689/1380] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:38.715518 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a7818-ab7b-4989-bd23-88872f9f17e6 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:38.716051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a7818-ab7b-4989-bd23-88872f9f17e6 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:38.716051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a7818-ab7b-4989-bd23-88872f9f17e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:38.807893 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-9c5a7818-ab7b-4989-bd23-88872f9f17e6 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:38.809320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-9c5a7818-ab7b-4989-bd23-88872f9f17e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:38.914090 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:38.933955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 692/1381] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:38.981442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efe87a-97d5-4fbc-8397-094a2e19e62c None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:38.983751 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b3efe87a-97d5-4fbc-8397-094a2e19e62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:39.057443 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:39.164349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efe87a-97d5-4fbc-8397-094a2e19e62c None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:39.294993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d32fd-c676-43de-86a8-9298491e6d30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:39.295830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d32fd-c676-43de-86a8-9298491e6d30 service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:39.296226 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d32fd-c676-43de-86a8-9298491e6d30 service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:39.296641 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d32fd-c676-43de-86a8-9298491e6d30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:39.298441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-093d32fd-c676-43de-86a8-9298491e6d30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:39.324188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:39.401870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d32fd-c676-43de-86a8-9298491e6d30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:39.403831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d32fd-c676-43de-86a8-9298491e6d30 service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:39.425721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 690/1382] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:05:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:05:41.766120 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24acf3-cd55-4a04-9ed6-fa19d47f1293 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:41.766965 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24acf3-cd55-4a04-9ed6-fa19d47f1293 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:41.766965 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24acf3-cd55-4a04-9ed6-fa19d47f1293 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:41.768286 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 693/1383] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:41.774195 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293d7d6-bffa-4d99-8ca5-6547fa6082dd None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:41.774771 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293d7d6-bffa-4d99-8ca5-6547fa6082dd None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:41.774771 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293d7d6-bffa-4d99-8ca5-6547fa6082dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:41.851024 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-3293d7d6-bffa-4d99-8ca5-6547fa6082dd None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:41.852924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-3293d7d6-bffa-4d99-8ca5-6547fa6082dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:41.966489 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:41.970275 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 691/1384] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:42.017888 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24acf3-cd55-4a04-9ed6-fa19d47f1293 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:42.019560 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ad24acf3-cd55-4a04-9ed6-fa19d47f1293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:42.187011 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24acf3-cd55-4a04-9ed6-fa19d47f1293 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:42.254180 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:42.328158 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e602339-8a17-4b2a-8634-ebe75246ef48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:42.328796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e602339-8a17-4b2a-8634-ebe75246ef48 service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:42.329029 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e602339-8a17-4b2a-8634-ebe75246ef48 service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:42.329323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e602339-8a17-4b2a-8634-ebe75246ef48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:42.331128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4e602339-8a17-4b2a-8634-ebe75246ef48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:42.435486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e602339-8a17-4b2a-8634-ebe75246ef48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:42.437199 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e602339-8a17-4b2a-8634-ebe75246ef48 service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:42.458789 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 694/1385] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:05:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:05:44.545589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32510383-5446-4252-8352-892b525084e3 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:44.546160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32510383-5446-4252-8352-892b525084e3 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:44.546160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32510383-5446-4252-8352-892b525084e3 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:44.547562 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 692/1386] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:44.554292 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bbf8f-58bf-40e6-a5b6-d919417f1551 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:44.555810 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bbf8f-58bf-40e6-a5b6-d919417f1551 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:44.555810 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bbf8f-58bf-40e6-a5b6-d919417f1551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:44.623184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-773bbf8f-58bf-40e6-a5b6-d919417f1551 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:44.624260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-773bbf8f-58bf-40e6-a5b6-d919417f1551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:44.653033 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:44.729757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 695/1387] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:44.788876 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32510383-5446-4252-8352-892b525084e3 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:44.790491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-32510383-5446-4252-8352-892b525084e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:44.951777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32510383-5446-4252-8352-892b525084e3 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:45.039675 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:45.084835 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf134d5c-a58e-4a06-a5b5-c1a549383674 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:45.085481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf134d5c-a58e-4a06-a5b5-c1a549383674 service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:45.085775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf134d5c-a58e-4a06-a5b5-c1a549383674 service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:45.086054 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf134d5c-a58e-4a06-a5b5-c1a549383674 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:45.087771 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-cf134d5c-a58e-4a06-a5b5-c1a549383674 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:45.192027 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf134d5c-a58e-4a06-a5b5-c1a549383674 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:45.194010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf134d5c-a58e-4a06-a5b5-c1a549383674 service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:45.216910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 693/1388] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:05:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:05:47.487037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6c5a4-169d-4d63-82f3-4f929f052826 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:47.487635 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6c5a4-169d-4d63-82f3-4f929f052826 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:47.487635 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6c5a4-169d-4d63-82f3-4f929f052826 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:47.488987 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 696/1389] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:47.494923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d4d3a-5976-4bfb-916b-9429c656bd6a None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:47.495391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d4d3a-5976-4bfb-916b-9429c656bd6a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:47.495488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d4d3a-5976-4bfb-916b-9429c656bd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:47.562658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-ab6d4d3a-5976-4bfb-916b-9429c656bd6a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:47.563929 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ab6d4d3a-5976-4bfb-916b-9429c656bd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:47.599388 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:47.664770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 694/1390] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:47.709723 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6c5a4-169d-4d63-82f3-4f929f052826 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:47.711971 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-eaf6c5a4-169d-4d63-82f3-4f929f052826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:47.870057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6c5a4-169d-4d63-82f3-4f929f052826 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:47.879361 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:47.998843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0453a3-46e1-467f-a64a-a570672cfadb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:47.999504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0453a3-46e1-467f-a64a-a570672cfadb service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:47.999797 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0453a3-46e1-467f-a64a-a570672cfadb service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:48.000078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0453a3-46e1-467f-a64a-a570672cfadb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:48.001736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-dc0453a3-46e1-467f-a64a-a570672cfadb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:48.103295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0453a3-46e1-467f-a64a-a570672cfadb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:48.104941 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0453a3-46e1-467f-a64a-a570672cfadb service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:48.124178 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 697/1391] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:05:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:05:48.803016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ded009-b588-44ad-8613-710bcd0ba5ed None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:48.803796 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ded009-b588-44ad-8613-710bcd0ba5ed None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:48.803796 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ded009-b588-44ad-8613-710bcd0ba5ed None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:48.805534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 695/1392] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:05:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:48.811416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0d00-5f6b-4336-8032-0887d32d2d8e None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:48.811922 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0d00-5f6b-4336-8032-0887d32d2d8e None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:48.812034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0d00-5f6b-4336-8032-0887d32d2d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:48.828037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:48.885323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-263a0d00-5f6b-4336-8032-0887d32d2d8e None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:48.886579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-263a0d00-5f6b-4336-8032-0887d32d2d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:48.989026 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 698/1393] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:49.046820 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5641d34-c0f7-42f8-8e8f-34d3f0519098 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:49.047197 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5641d34-c0f7-42f8-8e8f-34d3f0519098 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:49.047548 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5641d34-c0f7-42f8-8e8f-34d3f0519098 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:49.049036 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 696/1394] 10.222.0.83 () {54 vars in 961 bytes} [Mon Jul 18 15:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:49.054477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b3c99-0da4-4be3-b92e-e6c784567d85 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:49.055271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b3c99-0da4-4be3-b92e-e6c784567d85 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:49.055554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b3c99-0da4-4be3-b92e-e6c784567d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:49.141825 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-998b3c99-0da4-4be3-b92e-e6c784567d85 None None] MFA Rules not processed for user `ef8dcb0a0f2c4a2d955544324213739c`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:49.143089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-998b3c99-0da4-4be3-b92e-e6c784567d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:49.180625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:49.224792 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 699/1395] 10.222.0.83 () {58 vars in 1058 bytes} [Mon Jul 18 15:05:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:49.233634 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5641d34-c0f7-42f8-8e8f-34d3f0519098 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:49.235453 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-f5641d34-c0f7-42f8-8e8f-34d3f0519098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:49.355819 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5641d34-c0f7-42f8-8e8f-34d3f0519098 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:49.425393 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:49.439655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badb21b-2fab-4891-bcad-c585269ba8b6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8dcb0a0f2c4a2d955544324213739c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:49.440376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badb21b-2fab-4891-bcad-c585269ba8b6 service glance] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:49.440673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badb21b-2fab-4891-bcad-c585269ba8b6 service glance] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:49.440956 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badb21b-2fab-4891-bcad-c585269ba8b6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:49.442685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6badb21b-2fab-4891-bcad-c585269ba8b6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:49.561475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badb21b-2fab-4891-bcad-c585269ba8b6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:49.562972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badb21b-2fab-4891-bcad-c585269ba8b6 service glance] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:49.582317 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 697/1396] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:05:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:05:50.453072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1199-a27a-473f-8168-ec1288a1b77a None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:50.453637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1199-a27a-473f-8168-ec1288a1b77a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:50.453637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1199-a27a-473f-8168-ec1288a1b77a None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:50.455014 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 700/1397] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 15:05:50.460419 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fcfc3-879c-48cf-b268-0cf4ab839579 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:50.460867 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fcfc3-879c-48cf-b268-0cf4ab839579 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:50.461148 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fcfc3-879c-48cf-b268-0cf4ab839579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:50.514920 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba06a3-0c3f-4f35-9525-186b5a670fd8 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:50.515211 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba06a3-0c3f-4f35-9525-186b5a670fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:50.515492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba06a3-0c3f-4f35-9525-186b5a670fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:50.529610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-a20fcfc3-879c-48cf-b268-0cf4ab839579 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:50.530877 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a20fcfc3-879c-48cf-b268-0cf4ab839579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:50.576432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:50.583489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-00ba06a3-0c3f-4f35-9525-186b5a670fd8 None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:50.584765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-00ba06a3-0c3f-4f35-9525-186b5a670fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:50.635116 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 698/1398] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:05:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:50.677198 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 701/1399] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Jul 18 15:05:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:05:50.682497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fcfc3-879c-48cf-b268-0cf4ab839579 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:50.683742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a20fcfc3-879c-48cf-b268-0cf4ab839579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:50.722967 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1e92d-760c-41ce-80fd-3b1e229854dc None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:50.723268 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1e92d-760c-41ce-80fd-3b1e229854dc None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:50.723550 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1e92d-760c-41ce-80fd-3b1e229854dc None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:50.724954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 702/1400] 10.222.0.83 () {54 vars in 943 bytes} [Mon Jul 18 15:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:50.732383 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d234f-6f4c-4ca2-a105-5bfe18b263b1 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:50.732667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d234f-6f4c-4ca2-a105-5bfe18b263b1 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:50.732942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d234f-6f4c-4ca2-a105-5bfe18b263b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:50.792264 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:50.807846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fcfc3-879c-48cf-b268-0cf4ab839579 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:50.811894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-1f0d234f-6f4c-4ca2-a105-5bfe18b263b1 None None] MFA Rules not processed for user `f88dfe620ff04ffe88088a8a23c7d6e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:50.813146 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1f0d234f-6f4c-4ca2-a105-5bfe18b263b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:50.866504 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:50.924010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 703/1401] 10.222.0.83 () {58 vars in 1040 bytes} [Mon Jul 18 15:05:50 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:05:50.928556 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10376754-3f28-4e1a-9421-143c916f0800 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:50.929223 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10376754-3f28-4e1a-9421-143c916f0800 service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:50.930043 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10376754-3f28-4e1a-9421-143c916f0800 service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:50.931026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10376754-3f28-4e1a-9421-143c916f0800 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:50.933125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-10376754-3f28-4e1a-9421-143c916f0800 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:50.933844 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d234f-6f4c-4ca2-a105-5bfe18b263b1 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:50.943400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d234f-6f4c-4ca2-a105-5bfe18b263b1 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:51.027784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10376754-3f28-4e1a-9421-143c916f0800 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:51.029419 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10376754-3f28-4e1a-9421-143c916f0800 service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:51.049832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 699/1402] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:05:51.053884 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:51.060191 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5aac13-2e29-4521-af9a-9ad390803cca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:51.060847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5aac13-2e29-4521-af9a-9ad390803cca service swift] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:51.061133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5aac13-2e29-4521-af9a-9ad390803cca service swift] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:51.061414 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5aac13-2e29-4521-af9a-9ad390803cca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:51.063276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ec5aac13-2e29-4521-af9a-9ad390803cca service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:51.151748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5aac13-2e29-4521-af9a-9ad390803cca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:51.153387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5aac13-2e29-4521-af9a-9ad390803cca service swift] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:51.162153 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 704/1403] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:05:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 15:05:53.466091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dda1b-d3d0-44ba-9c64-b2e542a78781 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:53.466746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dda1b-d3d0-44ba-9c64-b2e542a78781 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:53.466746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dda1b-d3d0-44ba-9c64-b2e542a78781 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:53.468042 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 700/1404] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:53.473665 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff609a9-c214-49ce-b705-de3c77903ffe None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:53.474223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff609a9-c214-49ce-b705-de3c77903ffe None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:53.474335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff609a9-c214-49ce-b705-de3c77903ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:53.552970 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-bff609a9-c214-49ce-b705-de3c77903ffe None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:53.556675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-bff609a9-c214-49ce-b705-de3c77903ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:53.656480 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:53.673316 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 705/1405] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:05:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:05:53.720268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59dda1b-d3d0-44ba-9c64-b2e542a78781 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:53.721824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a59dda1b-d3d0-44ba-9c64-b2e542a78781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:53.810984 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:53.901608 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59dda1b-d3d0-44ba-9c64-b2e542a78781 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:54.040720 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8c769-d986-453f-8bfc-ee3049d6009a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:54.041400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8c769-d986-453f-8bfc-ee3049d6009a service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:54.041667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8c769-d986-453f-8bfc-ee3049d6009a service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:54.041850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8c769-d986-453f-8bfc-ee3049d6009a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:54.043643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-fff8c769-d986-453f-8bfc-ee3049d6009a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:54.084153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:54.166288 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8c769-d986-453f-8bfc-ee3049d6009a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:54.167952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8c769-d986-453f-8bfc-ee3049d6009a service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:54.191288 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 701/1406] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:05:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:05:56.444488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8c7c4-6e0b-4598-88d7-62ef7d5281ea None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:56.445677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8c7c4-6e0b-4598-88d7-62ef7d5281ea None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:56.445677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8c7c4-6e0b-4598-88d7-62ef7d5281ea None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:56.447331 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 706/1407] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:05:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:05:56.452585 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53470afc-c3ec-48f9-ae34-df48525e1f0f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:56.453061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53470afc-c3ec-48f9-ae34-df48525e1f0f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:56.453170 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53470afc-c3ec-48f9-ae34-df48525e1f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:56.531597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-53470afc-c3ec-48f9-ae34-df48525e1f0f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:05:56.532829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-53470afc-c3ec-48f9-ae34-df48525e1f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:56.652830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 702/1408] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:05:56.699334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8c7c4-6e0b-4598-88d7-62ef7d5281ea None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:05:56.701254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-13d8c7c4-6e0b-4598-88d7-62ef7d5281ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:56.880157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8c7c4-6e0b-4598-88d7-62ef7d5281ea None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:05:56.958233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:05:57.026616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7526823-d789-4e26-8f6c-993b5c013d3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:05:57.027492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7526823-d789-4e26-8f6c-993b5c013d3c service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:05:57.027892 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7526823-d789-4e26-8f6c-993b5c013d3c service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:05:57.028245 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7526823-d789-4e26-8f6c-993b5c013d3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:05:57.030147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e7526823-d789-4e26-8f6c-993b5c013d3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:05:57.136213 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7526823-d789-4e26-8f6c-993b5c013d3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:05:57.137875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7526823-d789-4e26-8f6c-993b5c013d3c service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:05:57.158229 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 707/1409] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:05:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:06.650914 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=98 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:07.155546 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=67 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:15.564988 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a45ec5-845e-433b-9508-8933e86fffc8 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:15.565619 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a45ec5-845e-433b-9508-8933e86fffc8 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:15.565619 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a45ec5-845e-433b-9508-8933e86fffc8 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:15.567041 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 703/1410] 10.222.0.83 () {54 vars in 918 bytes} [Mon Jul 18 15:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:15.573401 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24983ffb-ae00-471f-a67a-b75dbc196d46 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:15.573925 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24983ffb-ae00-471f-a67a-b75dbc196d46 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:15.574041 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24983ffb-ae00-471f-a67a-b75dbc196d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:15.654893 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-24983ffb-ae00-471f-a67a-b75dbc196d46 None None] MFA Rules not processed for user `76cc0ae48b5a49c6a7a1c11a08c389e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:15.656108 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-24983ffb-ae00-471f-a67a-b75dbc196d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:15.769516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 708/1411] 10.222.0.83 () {58 vars in 1015 bytes} [Mon Jul 18 15:06:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:15.784967 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae8d11-1b63-4f64-adf0-eaad22bb49e2 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:15.785257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae8d11-1b63-4f64-adf0-eaad22bb49e2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:15.785539 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae8d11-1b63-4f64-adf0-eaad22bb49e2 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:15.786965 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 704/1412] 10.222.0.83 () {54 vars in 918 bytes} [Mon Jul 18 15:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:15.792353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475dc15-00ec-469d-b946-7a4e1b01cba2 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:15.792645 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475dc15-00ec-469d-b946-7a4e1b01cba2 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:15.792932 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475dc15-00ec-469d-b946-7a4e1b01cba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:15.869617 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:15.872357 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-e475dc15-00ec-469d-b946-7a4e1b01cba2 None None] MFA Rules not processed for user `7e1355d18fb049468651b3675c217d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:15.873609 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e475dc15-00ec-469d-b946-7a4e1b01cba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:15.994443 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 709/1413] 10.222.0.83 () {58 vars in 1015 bytes} [Mon Jul 18 15:06:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:18.556443 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715e67c-eb37-48c0-bbbc-87fee159934e None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:18.557081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715e67c-eb37-48c0-bbbc-87fee159934e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:18.557081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715e67c-eb37-48c0-bbbc-87fee159934e None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:18.558435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 15:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:18.563990 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926fca7-23cc-4d3f-a5e9-df95e9e7c1f8 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:18.564451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926fca7-23cc-4d3f-a5e9-df95e9e7c1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:18.564566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926fca7-23cc-4d3f-a5e9-df95e9e7c1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:18.632483 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-5926fca7-23cc-4d3f-a5e9-df95e9e7c1f8 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:18.633726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-5926fca7-23cc-4d3f-a5e9-df95e9e7c1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:18.689687 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:18.734312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 15:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:18.782774 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715e67c-eb37-48c0-bbbc-87fee159934e None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:18.784189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-f715e67c-eb37-48c0-bbbc-87fee159934e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:18.937795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715e67c-eb37-48c0-bbbc-87fee159934e None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:19.015147 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:19.056010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bacd2-5fc3-4b5f-a397-3feda613542c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:19.056665 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bacd2-5fc3-4b5f-a397-3feda613542c service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:19.056956 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bacd2-5fc3-4b5f-a397-3feda613542c service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:19.057233 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bacd2-5fc3-4b5f-a397-3feda613542c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:19.058952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-052bacd2-5fc3-4b5f-a397-3feda613542c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:19.150983 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bacd2-5fc3-4b5f-a397-3feda613542c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:19.152616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bacd2-5fc3-4b5f-a397-3feda613542c service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:19.170825 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 706/1416] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:20.118759 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd46676-4afe-4848-bfa7-45ee6f825192 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:20.119642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd46676-4afe-4848-bfa7-45ee6f825192 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:20.119642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd46676-4afe-4848-bfa7-45ee6f825192 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:20.120738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 15:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:20.126107 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823c78b-b723-45eb-a9f8-84dc1c958228 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:20.126658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823c78b-b723-45eb-a9f8-84dc1c958228 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:20.126771 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823c78b-b723-45eb-a9f8-84dc1c958228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:20.206818 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-c823c78b-b723-45eb-a9f8-84dc1c958228 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:20.207961 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c823c78b-b723-45eb-a9f8-84dc1c958228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:20.236597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:20.310505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 15:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:20.467502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd46676-4afe-4848-bfa7-45ee6f825192 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:20.468764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-5fd46676-4afe-4848-bfa7-45ee6f825192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:20.622439 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd46676-4afe-4848-bfa7-45ee6f825192 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:20.664447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:20.742267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967afba-fcc1-4b38-b946-6d16e54e5667 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:20.742956 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967afba-fcc1-4b38-b946-6d16e54e5667 service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:20.743244 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967afba-fcc1-4b38-b946-6d16e54e5667 service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:20.743547 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967afba-fcc1-4b38-b946-6d16e54e5667 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:20.745222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-9967afba-fcc1-4b38-b946-6d16e54e5667 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:20.842565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967afba-fcc1-4b38-b946-6d16e54e5667 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:20.844226 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967afba-fcc1-4b38-b946-6d16e54e5667 service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:20.866328 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 712/1419] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:21.782375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd04815-b2e4-40c1-92dc-089dc8e4ba86 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:21.783136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd04815-b2e4-40c1-92dc-089dc8e4ba86 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:21.783136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd04815-b2e4-40c1-92dc-089dc8e4ba86 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:21.785559 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 15:06:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:21.790985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852c1f9-14b3-4d1b-8272-ecab7a6911c6 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:21.791453 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852c1f9-14b3-4d1b-8272-ecab7a6911c6 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:21.791577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852c1f9-14b3-4d1b-8272-ecab7a6911c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:21.845275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:21.868446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-4852c1f9-14b3-4d1b-8272-ecab7a6911c6 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:21.869672 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4852c1f9-14b3-4d1b-8272-ecab7a6911c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:21.971025 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 15:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:22.128003 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd04815-b2e4-40c1-92dc-089dc8e4ba86 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:22.129456 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0cd04815-b2e4-40c1-92dc-089dc8e4ba86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:22.285603 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd04815-b2e4-40c1-92dc-089dc8e4ba86 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:22.299010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:22.401417 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406dad5-c02a-4313-9c96-22c87033ed01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:22.401990 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406dad5-c02a-4313-9c96-22c87033ed01 service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:22.402277 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406dad5-c02a-4313-9c96-22c87033ed01 service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:22.402577 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406dad5-c02a-4313-9c96-22c87033ed01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:22.404266 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-5406dad5-c02a-4313-9c96-22c87033ed01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:22.499984 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406dad5-c02a-4313-9c96-22c87033ed01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:22.501597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406dad5-c02a-4313-9c96-22c87033ed01 service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:22.520153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 709/1422] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:23.401614 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c964574-5f51-4842-9347-f207e4d5f102 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:23.402265 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c964574-5f51-4842-9347-f207e4d5f102 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:23.402265 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c964574-5f51-4842-9347-f207e4d5f102 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:23.403732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 15:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:23.409282 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895270b8-75fa-4ce1-a57c-b2db33be762b None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:23.409931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895270b8-75fa-4ce1-a57c-b2db33be762b None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:23.409931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895270b8-75fa-4ce1-a57c-b2db33be762b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:23.431407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:23.491212 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-895270b8-75fa-4ce1-a57c-b2db33be762b None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:23.492455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-895270b8-75fa-4ce1-a57c-b2db33be762b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:23.590592 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 15:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:06:23.744986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c964574-5f51-4842-9347-f207e4d5f102 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:23.746506 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-0c964574-5f51-4842-9347-f207e4d5f102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:23.871141 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:23.909273 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c964574-5f51-4842-9347-f207e4d5f102 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:24.028921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ca5ce-a344-4fdf-86c7-71eaa1288d2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:24.029486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ca5ce-a344-4fdf-86c7-71eaa1288d2e service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:24.029778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ca5ce-a344-4fdf-86c7-71eaa1288d2e service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:24.030057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ca5ce-a344-4fdf-86c7-71eaa1288d2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:24.031836 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-2a4ca5ce-a344-4fdf-86c7-71eaa1288d2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:24.112018 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:24.126248 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ca5ce-a344-4fdf-86c7-71eaa1288d2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:24.127924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ca5ce-a344-4fdf-86c7-71eaa1288d2e service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:24.147248 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 715/1425] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:25.059937 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d387952-b2a3-4141-bf52-ce25bdb27552 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:25.060551 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d387952-b2a3-4141-bf52-ce25bdb27552 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:25.060551 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d387952-b2a3-4141-bf52-ce25bdb27552 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:25.061874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 15:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:25.067031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3e131-f95a-4545-a5ec-6fb25715c7c7 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:25.067446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3e131-f95a-4545-a5ec-6fb25715c7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:25.067603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3e131-f95a-4545-a5ec-6fb25715c7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:25.145769 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-fcd3e131-f95a-4545-a5ec-6fb25715c7c7 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:25.147629 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-fcd3e131-f95a-4545-a5ec-6fb25715c7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:25.246145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 15:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:25.400140 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d387952-b2a3-4141-bf52-ce25bdb27552 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:25.401683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7d387952-b2a3-4141-bf52-ce25bdb27552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:25.488748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:25.556663 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d387952-b2a3-4141-bf52-ce25bdb27552 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:25.675807 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88f83a-9979-49a4-ab81-c0884392359d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:25.676767 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88f83a-9979-49a4-ab81-c0884392359d service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:25.677037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88f83a-9979-49a4-ab81-c0884392359d service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:25.677309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88f83a-9979-49a4-ab81-c0884392359d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:25.679760 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0a88f83a-9979-49a4-ab81-c0884392359d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:25.728803 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:25.772024 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88f83a-9979-49a4-ab81-c0884392359d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:25.773523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88f83a-9979-49a4-ab81-c0884392359d service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:25.791974 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 712/1428] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:26.708169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60339e7a-a5c6-43aa-b933-bfd13c2f8999 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:26.708815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60339e7a-a5c6-43aa-b933-bfd13c2f8999 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:26.708985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60339e7a-a5c6-43aa-b933-bfd13c2f8999 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:26.710335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 15:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:26.715629 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657f10e-1243-462d-8e24-2a198218bc8b None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:26.716135 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657f10e-1243-462d-8e24-2a198218bc8b None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:26.716237 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657f10e-1243-462d-8e24-2a198218bc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:26.792095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-7657f10e-1243-462d-8e24-2a198218bc8b None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:26.793336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7657f10e-1243-462d-8e24-2a198218bc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:26.891634 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 15:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:27.046646 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60339e7a-a5c6-43aa-b933-bfd13c2f8999 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:27.048232 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-60339e7a-a5c6-43aa-b933-bfd13c2f8999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:27.102020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:27.200390 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60339e7a-a5c6-43aa-b933-bfd13c2f8999 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:27.318007 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d315e5a-3ba8-48d3-80a9-7d8876a8f87e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:27.318820 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315e5a-3ba8-48d3-80a9-7d8876a8f87e service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:27.318948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315e5a-3ba8-48d3-80a9-7d8876a8f87e service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:27.319234 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315e5a-3ba8-48d3-80a9-7d8876a8f87e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:27.320909 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-6d315e5a-3ba8-48d3-80a9-7d8876a8f87e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:27.328239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:27.417455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d315e5a-3ba8-48d3-80a9-7d8876a8f87e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:27.419189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d315e5a-3ba8-48d3-80a9-7d8876a8f87e service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:27.438360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 718/1431] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:28.412911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc305a-8362-4888-a57a-d8dab05c2ecd None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:28.413495 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc305a-8362-4888-a57a-d8dab05c2ecd None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:28.413495 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc305a-8362-4888-a57a-d8dab05c2ecd None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:28.414985 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 15:06:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:28.420236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5fdeb-bbf3-49cf-a5cc-09bf0088265d None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:28.420677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5fdeb-bbf3-49cf-a5cc-09bf0088265d None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:28.420793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5fdeb-bbf3-49cf-a5cc-09bf0088265d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:28.500106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-4ed5fdeb-bbf3-49cf-a5cc-09bf0088265d None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:28.501349 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4ed5fdeb-bbf3-49cf-a5cc-09bf0088265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:28.571850 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:28.604400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 15:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:28.760445 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc305a-8362-4888-a57a-d8dab05c2ecd None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:28.761978 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d0bc305a-8362-4888-a57a-d8dab05c2ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:28.781648 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:28.915401 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc305a-8362-4888-a57a-d8dab05c2ecd None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:29.007440 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:29.035541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdde47-d22d-49e7-a118-213c63e8b288 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:29.036102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdde47-d22d-49e7-a118-213c63e8b288 service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:29.036376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdde47-d22d-49e7-a118-213c63e8b288 service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:29.036689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdde47-d22d-49e7-a118-213c63e8b288 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:29.038442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c3bdde47-d22d-49e7-a118-213c63e8b288 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:29.129299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdde47-d22d-49e7-a118-213c63e8b288 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:29.131409 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdde47-d22d-49e7-a118-213c63e8b288 service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:29.150583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 715/1434] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:30.028656 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43aa82-b803-4143-860a-a2e38c556d0a None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:30.029295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43aa82-b803-4143-860a-a2e38c556d0a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:30.029295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43aa82-b803-4143-860a-a2e38c556d0a None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:30.030675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 15:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:30.036323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab008261-889a-48fc-a66a-4b5450b85266 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:30.036838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab008261-889a-48fc-a66a-4b5450b85266 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:30.037042 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab008261-889a-48fc-a66a-4b5450b85266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:30.113353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-ab008261-889a-48fc-a66a-4b5450b85266 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:30.114609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ab008261-889a-48fc-a66a-4b5450b85266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:30.157207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:30.214231 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 15:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:30.371113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43aa82-b803-4143-860a-a2e38c556d0a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:30.372682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-bb43aa82-b803-4143-860a-a2e38c556d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:30.521576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43aa82-b803-4143-860a-a2e38c556d0a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:30.576846 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:30.638266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6aea55-ebdc-409a-a81d-7d82ddf8dd24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:30.638957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6aea55-ebdc-409a-a81d-7d82ddf8dd24 service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:30.639248 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6aea55-ebdc-409a-a81d-7d82ddf8dd24 service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:30.639526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6aea55-ebdc-409a-a81d-7d82ddf8dd24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:30.641206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-fb6aea55-ebdc-409a-a81d-7d82ddf8dd24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:30.732434 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6aea55-ebdc-409a-a81d-7d82ddf8dd24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:30.734100 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6aea55-ebdc-409a-a81d-7d82ddf8dd24 service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:30.752682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 721/1437] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:31.654091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b1288-dec4-4b0b-a06e-d66346b4aab5 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:31.654749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b1288-dec4-4b0b-a06e-d66346b4aab5 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:31.654749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b1288-dec4-4b0b-a06e-d66346b4aab5 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:31.656113 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 15:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:31.661463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e83d84-f6f8-4ee2-b050-c9a54bbdfa06 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:31.662638 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e83d84-f6f8-4ee2-b050-c9a54bbdfa06 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:31.662740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e83d84-f6f8-4ee2-b050-c9a54bbdfa06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:31.740200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-a8e83d84-f6f8-4ee2-b050-c9a54bbdfa06 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:31.741478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a8e83d84-f6f8-4ee2-b050-c9a54bbdfa06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:31.743775 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:31.842139 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 15:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:31.998131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b1288-dec4-4b0b-a06e-d66346b4aab5 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:31.999755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6e9b1288-dec4-4b0b-a06e-d66346b4aab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:32.161733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b1288-dec4-4b0b-a06e-d66346b4aab5 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:32.188180 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:32.276923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca8f0e-0dad-4d0a-af5f-e2d0f11f75c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:32.277588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca8f0e-0dad-4d0a-af5f-e2d0f11f75c9 service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:32.277877 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca8f0e-0dad-4d0a-af5f-e2d0f11f75c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:32.278156 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca8f0e-0dad-4d0a-af5f-e2d0f11f75c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:32.279934 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d8ca8f0e-0dad-4d0a-af5f-e2d0f11f75c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:32.372497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca8f0e-0dad-4d0a-af5f-e2d0f11f75c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:32.374136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca8f0e-0dad-4d0a-af5f-e2d0f11f75c9 service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:32.393524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 718/1440] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:34.400340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afc620-eb58-4ddd-9754-80bab5a44604 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:34.400960 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afc620-eb58-4ddd-9754-80bab5a44604 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:34.400960 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afc620-eb58-4ddd-9754-80bab5a44604 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:34.402300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 15:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:34.409492 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee3554-d74d-4d25-9cde-490d7e28c93f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:34.410013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee3554-d74d-4d25-9cde-490d7e28c93f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:34.410108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee3554-d74d-4d25-9cde-490d7e28c93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:34.460309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:34.488615 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-f3ee3554-d74d-4d25-9cde-490d7e28c93f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:34.491450 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-f3ee3554-d74d-4d25-9cde-490d7e28c93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:34.592291 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 15:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:06:34.633526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afc620-eb58-4ddd-9754-80bab5a44604 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:34.635157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-92afc620-eb58-4ddd-9754-80bab5a44604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:34.753122 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afc620-eb58-4ddd-9754-80bab5a44604 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:34.789363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:34.840606 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f4278-04e2-432d-bf18-3d2aaa2c9825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:34.841258 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f4278-04e2-432d-bf18-3d2aaa2c9825 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:34.841546 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f4278-04e2-432d-bf18-3d2aaa2c9825 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:34.841819 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f4278-04e2-432d-bf18-3d2aaa2c9825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:34.843534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-440f4278-04e2-432d-bf18-3d2aaa2c9825 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:34.967595 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f4278-04e2-432d-bf18-3d2aaa2c9825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:34.969442 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f4278-04e2-432d-bf18-3d2aaa2c9825 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:34.993752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 724/1443] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:37.302666 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61283488-f3c9-4066-a64f-847be9dae186 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:37.303317 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61283488-f3c9-4066-a64f-847be9dae186 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:37.303317 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61283488-f3c9-4066-a64f-847be9dae186 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:37.304954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 15:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 15:06:37.310351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9526e3-7284-44f1-8f21-cd78b47107a5 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:37.310892 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9526e3-7284-44f1-8f21-cd78b47107a5 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:37.311003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9526e3-7284-44f1-8f21-cd78b47107a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:37.387560 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-9a9526e3-7284-44f1-8f21-cd78b47107a5 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:37.388760 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-9a9526e3-7284-44f1-8f21-cd78b47107a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:37.403458 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:37.496126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 15:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:37.535895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61283488-f3c9-4066-a64f-847be9dae186 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:37.537844 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-61283488-f3c9-4066-a64f-847be9dae186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:37.643634 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61283488-f3c9-4066-a64f-847be9dae186 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:37.656408 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:37.720148 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c8a8d-9d82-4d75-afdf-7635a3331464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:37.720798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c8a8d-9d82-4d75-afdf-7635a3331464 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:37.721086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c8a8d-9d82-4d75-afdf-7635a3331464 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:37.721370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c8a8d-9d82-4d75-afdf-7635a3331464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:37.723109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-f12c8a8d-9d82-4d75-afdf-7635a3331464 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:37.841919 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c8a8d-9d82-4d75-afdf-7635a3331464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:37.843396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c8a8d-9d82-4d75-afdf-7635a3331464 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:37.862972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 721/1446] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:40.354744 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02170cd6-bb15-44f0-b8cb-bb71b95fe477 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:40.355468 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02170cd6-bb15-44f0-b8cb-bb71b95fe477 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:40.355468 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02170cd6-bb15-44f0-b8cb-bb71b95fe477 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:40.356704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 15:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:40.362250 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a74fe-ee59-4282-a70a-1e24d36261ce None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:40.362770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a74fe-ee59-4282-a70a-1e24d36261ce None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:40.362888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a74fe-ee59-4282-a70a-1e24d36261ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:40.414208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:40.433642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-9f8a74fe-ee59-4282-a70a-1e24d36261ce None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:40.435113 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9f8a74fe-ee59-4282-a70a-1e24d36261ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:40.533795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 15:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:40.575178 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02170cd6-bb15-44f0-b8cb-bb71b95fe477 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:40.576719 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-02170cd6-bb15-44f0-b8cb-bb71b95fe477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:40.694356 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02170cd6-bb15-44f0-b8cb-bb71b95fe477 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:40.736022 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:40.772275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fb38a-ca8e-4db6-87fd-fafc301a3c62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:40.772923 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fb38a-ca8e-4db6-87fd-fafc301a3c62 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:40.773211 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fb38a-ca8e-4db6-87fd-fafc301a3c62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:40.773489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fb38a-ca8e-4db6-87fd-fafc301a3c62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:40.777312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c51fb38a-ca8e-4db6-87fd-fafc301a3c62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:40.888255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fb38a-ca8e-4db6-87fd-fafc301a3c62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:40.889736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fb38a-ca8e-4db6-87fd-fafc301a3c62 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:40.909501 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 727/1449] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:43.140664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2773a1-eb3f-48c3-b18a-ad26aec8cfc9 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:43.141109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2773a1-eb3f-48c3-b18a-ad26aec8cfc9 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:43.141234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2773a1-eb3f-48c3-b18a-ad26aec8cfc9 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:43.142652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 15:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:43.149057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a0679-ef4d-49ef-bf4a-b0fb82d47152 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:43.149523 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a0679-ef4d-49ef-bf4a-b0fb82d47152 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:43.149637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a0679-ef4d-49ef-bf4a-b0fb82d47152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:43.218154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-229a0679-ef4d-49ef-bf4a-b0fb82d47152 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:43.219322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-229a0679-ef4d-49ef-bf4a-b0fb82d47152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:43.241774 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:43.328095 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 15:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:43.368491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2773a1-eb3f-48c3-b18a-ad26aec8cfc9 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:43.370131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7c2773a1-eb3f-48c3-b18a-ad26aec8cfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:43.478617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2773a1-eb3f-48c3-b18a-ad26aec8cfc9 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:43.499298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:43.555323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1099f0-4c71-49f0-bbe1-23a4d7ae2824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:43.555980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1099f0-4c71-49f0-bbe1-23a4d7ae2824 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:43.556271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1099f0-4c71-49f0-bbe1-23a4d7ae2824 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:43.556553 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1099f0-4c71-49f0-bbe1-23a4d7ae2824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:43.558264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-bd1099f0-4c71-49f0-bbe1-23a4d7ae2824 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:43.670056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1099f0-4c71-49f0-bbe1-23a4d7ae2824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:43.671554 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1099f0-4c71-49f0-bbe1-23a4d7ae2824 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:43.690640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 724/1452] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:45.847402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86268a48-02c3-44e6-86a2-7700e1be9ce2 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:45.848048 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86268a48-02c3-44e6-86a2-7700e1be9ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:45.848048 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86268a48-02c3-44e6-86a2-7700e1be9ce2 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:45.849941 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 15:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:45.855424 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d240b-367f-46f3-83b7-35c2a7e390ba None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:45.855905 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d240b-367f-46f3-83b7-35c2a7e390ba None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:45.856017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d240b-367f-46f3-83b7-35c2a7e390ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:45.919282 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:45.921994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b57d240b-367f-46f3-83b7-35c2a7e390ba None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:45.923296 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b57d240b-367f-46f3-83b7-35c2a7e390ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:46.025709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 15:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:46.064720 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86268a48-02c3-44e6-86a2-7700e1be9ce2 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:46.066274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-86268a48-02c3-44e6-86a2-7700e1be9ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:46.174319 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86268a48-02c3-44e6-86a2-7700e1be9ce2 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:46.224869 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:46.250157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe479a-6d20-44c9-b2f1-aee11d8a6a94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:46.250824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe479a-6d20-44c9-b2f1-aee11d8a6a94 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:46.251113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe479a-6d20-44c9-b2f1-aee11d8a6a94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:46.251388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe479a-6d20-44c9-b2f1-aee11d8a6a94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:46.253061 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-5bbe479a-6d20-44c9-b2f1-aee11d8a6a94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:46.366277 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe479a-6d20-44c9-b2f1-aee11d8a6a94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:46.367786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe479a-6d20-44c9-b2f1-aee11d8a6a94 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:46.387784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 730/1455] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:48.616638 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d058819-e15f-457f-b183-c658e2fdb0f2 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:48.617215 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d058819-e15f-457f-b183-c658e2fdb0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:48.617215 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d058819-e15f-457f-b183-c658e2fdb0f2 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:48.618489 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 15:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:48.623897 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9a9fe-4589-4e08-9a7b-c4148c211b10 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:48.624326 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9a9fe-4589-4e08-9a7b-c4148c211b10 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:48.624452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9a9fe-4589-4e08-9a7b-c4148c211b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:48.692857 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-e8a9a9fe-4589-4e08-9a7b-c4148c211b10 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:48.694093 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e8a9a9fe-4589-4e08-9a7b-c4148c211b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:48.725052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:48.795880 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 15:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:48.836022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d058819-e15f-457f-b183-c658e2fdb0f2 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:48.837595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-8d058819-e15f-457f-b183-c658e2fdb0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:48.945878 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d058819-e15f-457f-b183-c658e2fdb0f2 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:48.975408 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:49.023600 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08195394-674c-434e-b5b6-79e93b2e0e46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:49.024253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08195394-674c-434e-b5b6-79e93b2e0e46 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:49.024546 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08195394-674c-434e-b5b6-79e93b2e0e46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:49.024841 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08195394-674c-434e-b5b6-79e93b2e0e46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:49.026517 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-08195394-674c-434e-b5b6-79e93b2e0e46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:49.138650 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08195394-674c-434e-b5b6-79e93b2e0e46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:49.140197 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08195394-674c-434e-b5b6-79e93b2e0e46 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:49.160559 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 727/1458] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:51.305349 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb5870-f7c8-4b4a-8046-7d5bcea08cbc None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:51.305948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb5870-f7c8-4b4a-8046-7d5bcea08cbc None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:51.305948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb5870-f7c8-4b4a-8046-7d5bcea08cbc None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:51.307354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 15:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:51.312969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0e4d6-3948-4493-8019-c011915ed4bb None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:51.313425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0e4d6-3948-4493-8019-c011915ed4bb None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:51.313545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0e4d6-3948-4493-8019-c011915ed4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:51.502592 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-19d0e4d6-3948-4493-8019-c011915ed4bb None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:51.504397 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-19d0e4d6-3948-4493-8019-c011915ed4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:51.513580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:51.606526 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 15:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:51.644531 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb5870-f7c8-4b4a-8046-7d5bcea08cbc None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:51.646790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b2bb5870-f7c8-4b4a-8046-7d5bcea08cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:51.755368 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb5870-f7c8-4b4a-8046-7d5bcea08cbc None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:51.814838 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:51.855886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c660a45-0146-4cb3-a58e-39f5472b73e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:51.856616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c660a45-0146-4cb3-a58e-39f5472b73e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:51.856839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c660a45-0146-4cb3-a58e-39f5472b73e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:51.857119 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c660a45-0146-4cb3-a58e-39f5472b73e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:51.859034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4c660a45-0146-4cb3-a58e-39f5472b73e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:52.016722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c660a45-0146-4cb3-a58e-39f5472b73e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:52.018191 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c660a45-0146-4cb3-a58e-39f5472b73e3 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:52.040121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 733/1461] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:54.153589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196cf19-95fb-4c85-9cf2-ec1bfc4fe0fb None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:54.154199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196cf19-95fb-4c85-9cf2-ec1bfc4fe0fb None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:54.154199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196cf19-95fb-4c85-9cf2-ec1bfc4fe0fb None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:54.155579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 15:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:54.161773 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360df898-fa48-47c2-96b1-509a7005cbe4 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:54.162236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360df898-fa48-47c2-96b1-509a7005cbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:54.162335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360df898-fa48-47c2-96b1-509a7005cbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:54.231455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-360df898-fa48-47c2-96b1-509a7005cbe4 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:54.232711 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-360df898-fa48-47c2-96b1-509a7005cbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:54.273205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:54.337306 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 15:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:54.375938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196cf19-95fb-4c85-9cf2-ec1bfc4fe0fb None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:54.377476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7196cf19-95fb-4c85-9cf2-ec1bfc4fe0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:54.485918 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196cf19-95fb-4c85-9cf2-ec1bfc4fe0fb None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:54.524363 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:54.561671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a8960-767c-416e-9ce8-5fb886d22c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:54.562316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a8960-767c-416e-9ce8-5fb886d22c98 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:54.562636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a8960-767c-416e-9ce8-5fb886d22c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:54.562916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a8960-767c-416e-9ce8-5fb886d22c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:54.564594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-522a8960-767c-416e-9ce8-5fb886d22c98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:54.676773 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a8960-767c-416e-9ce8-5fb886d22c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:54.678220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a8960-767c-416e-9ce8-5fb886d22c98 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:54.697160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 730/1464] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:56.794062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de0c61-27f4-4b87-81f5-82cf641563d8 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:56.794699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de0c61-27f4-4b87-81f5-82cf641563d8 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:56.794699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de0c61-27f4-4b87-81f5-82cf641563d8 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:56.796076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 15:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:56.802827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba1cac-35f3-4bb1-80a6-0aac66c15085 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:56.803343 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba1cac-35f3-4bb1-80a6-0aac66c15085 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:56.803445 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba1cac-35f3-4bb1-80a6-0aac66c15085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:56.879282 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d3ba1cac-35f3-4bb1-80a6-0aac66c15085 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:56.880547 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d3ba1cac-35f3-4bb1-80a6-0aac66c15085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:56.898787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:56.984312 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 15:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:57.023930 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de0c61-27f4-4b87-81f5-82cf641563d8 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:57.025513 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-68de0c61-27f4-4b87-81f5-82cf641563d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:57.141188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de0c61-27f4-4b87-81f5-82cf641563d8 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:57.220513 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:57.231388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df34c0-16ab-4352-b0e3-dbedef0ac316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:57.232056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df34c0-16ab-4352-b0e3-dbedef0ac316 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:57.232517 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df34c0-16ab-4352-b0e3-dbedef0ac316 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:57.233773 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df34c0-16ab-4352-b0e3-dbedef0ac316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:57.235667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-51df34c0-16ab-4352-b0e3-dbedef0ac316 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:57.353219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df34c0-16ab-4352-b0e3-dbedef0ac316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:57.354733 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df34c0-16ab-4352-b0e3-dbedef0ac316 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:06:57.374519 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 736/1467] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:06:59.468638 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09004afe-480c-4c54-bd52-0d24d4dde18b None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:59.469269 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09004afe-480c-4c54-bd52-0d24d4dde18b None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:59.469269 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09004afe-480c-4c54-bd52-0d24d4dde18b None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:59.470598 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 15:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:06:59.476655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5faf7-cbcd-4331-8188-e659131bebbe None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:59.477150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5faf7-cbcd-4331-8188-e659131bebbe None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:59.477269 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5faf7-cbcd-4331-8188-e659131bebbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:59.543903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-d5c5faf7-cbcd-4331-8188-e659131bebbe None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:06:59.545154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d5c5faf7-cbcd-4331-8188-e659131bebbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:59.597886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:59.649121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 15:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:06:59.687052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09004afe-480c-4c54-bd52-0d24d4dde18b None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:06:59.688605 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-09004afe-480c-4c54-bd52-0d24d4dde18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:59.793833 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09004afe-480c-4c54-bd52-0d24d4dde18b None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:06:59.841795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:06:59.868891 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32061e81-ff31-467d-be5b-4a8636327f97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:06:59.869537 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32061e81-ff31-467d-be5b-4a8636327f97 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:06:59.869821 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32061e81-ff31-467d-be5b-4a8636327f97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:06:59.870159 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32061e81-ff31-467d-be5b-4a8636327f97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:06:59.872523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-32061e81-ff31-467d-be5b-4a8636327f97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:06:59.983996 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32061e81-ff31-467d-be5b-4a8636327f97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:06:59.985474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32061e81-ff31-467d-be5b-4a8636327f97 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:07:00.005444 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 733/1470] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:07:09.647208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=22 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:07:10.003798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=66 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:07:20.276382 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b553518-5fa6-4359-8584-6ea4e118e4e1 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:20.276999 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b553518-5fa6-4359-8584-6ea4e118e4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:20.276999 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b553518-5fa6-4359-8584-6ea4e118e4e1 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:20.278346 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 738/1471] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:07:20.284413 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0699113-b28f-443b-add1-e13c9eea4194 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:20.284902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0699113-b28f-443b-add1-e13c9eea4194 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:20.285015 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0699113-b28f-443b-add1-e13c9eea4194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:20.352492 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d0699113-b28f-443b-add1-e13c9eea4194 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:07:20.353719 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d0699113-b28f-443b-add1-e13c9eea4194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:20.449260 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 734/1472] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:07:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:07:20.498933 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da47327-059a-4bfd-9bf9-ca80227983df None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:20.499228 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da47327-059a-4bfd-9bf9-ca80227983df None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:20.499512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da47327-059a-4bfd-9bf9-ca80227983df None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:20.500892 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 739/1473] 10.222.0.83 () {54 vars in 961 bytes} [Mon Jul 18 15:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:07:20.506320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754eab27-66c7-4700-9889-5957b2260cb4 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:20.506650 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754eab27-66c7-4700-9889-5957b2260cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:20.506932 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754eab27-66c7-4700-9889-5957b2260cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:20.581444 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-754eab27-66c7-4700-9889-5957b2260cb4 None None] MFA Rules not processed for user `ef8dcb0a0f2c4a2d955544324213739c`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:07:20.582595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-754eab27-66c7-4700-9889-5957b2260cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:20.603842 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:07:20.659711 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 735/1474] 10.222.0.83 () {58 vars in 1058 bytes} [Mon Jul 18 15:07:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:07:20.667247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da47327-059a-4bfd-9bf9-ca80227983df None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:07:20.668781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-8da47327-059a-4bfd-9bf9-ca80227983df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:20.775704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da47327-059a-4bfd-9bf9-ca80227983df None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:07:20.850242 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d435c-c4d0-447c-8951-b9ac6322a15d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8dcb0a0f2c4a2d955544324213739c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:07:20.850838 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d435c-c4d0-447c-8951-b9ac6322a15d service glance] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:20.851119 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d435c-c4d0-447c-8951-b9ac6322a15d service glance] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:20.851397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d435c-c4d0-447c-8951-b9ac6322a15d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:20.853071 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e16d435c-c4d0-447c-8951-b9ac6322a15d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:20.909436 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:07:20.967357 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d435c-c4d0-447c-8951-b9ac6322a15d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:07:20.968828 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d435c-c4d0-447c-8951-b9ac6322a15d service glance] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:07:20.989309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 740/1475] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:07:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:07:23.223500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d9509-3f99-46eb-bb14-a5a95205180c None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:23.224058 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d9509-3f99-46eb-bb14-a5a95205180c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:23.224058 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d9509-3f99-46eb-bb14-a5a95205180c None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:23.225410 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 736/1476] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:07:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:07:23.230916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a5af2-3a58-4bc8-a966-d4278479287e None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:23.231373 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a5af2-3a58-4bc8-a966-d4278479287e None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:23.231506 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a5af2-3a58-4bc8-a966-d4278479287e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:23.300048 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-f20a5af2-3a58-4bc8-a966-d4278479287e None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:07:23.301312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f20a5af2-3a58-4bc8-a966-d4278479287e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:23.417428 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 741/1477] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:07:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:07:23.442791 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d9509-3f99-46eb-bb14-a5a95205180c None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:07:23.444338 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-341d9509-3f99-46eb-bb14-a5a95205180c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:23.625618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d9509-3f99-46eb-bb14-a5a95205180c None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:07:23.682425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:07:23.735056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d717f9-cac8-4b30-a046-950b624d62ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:07:23.735723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d717f9-cac8-4b30-a046-950b624d62ce admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:23.736068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d717f9-cac8-4b30-a046-950b624d62ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:23.736545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d717f9-cac8-4b30-a046-950b624d62ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:23.737398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d717f9-cac8-4b30-a046-950b624d62ce admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:07:23.739201 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: WARNING py.warnings [None req-e7d717f9-cac8-4b30-a046-950b624d62ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 15:07:23.739201 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: warnings.warn(msg) Jul 18 15:07:23.739201 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: Jul 18 15:07:23.740356 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d717f9-cac8-4b30-a046-950b624d62ce admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:07:23.760653 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 737/1478] 10.222.0.83 () {56 vars in 1108 bytes} [Mon Jul 18 15:07:23 2022] GET /identity/v3/projects => generated 2553 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:07:25.904007 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fe0f5-b4bc-418d-9da1-2743560f45e8 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:25.904579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fe0f5-b4bc-418d-9da1-2743560f45e8 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:25.904579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fe0f5-b4bc-418d-9da1-2743560f45e8 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:25.905980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 742/1479] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:07:25.911575 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a6a0a-2266-4560-96f9-2b22c2d8d39d None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:25.912019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a6a0a-2266-4560-96f9-2b22c2d8d39d None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:25.912125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a6a0a-2266-4560-96f9-2b22c2d8d39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:25.978152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-732a6a0a-2266-4560-96f9-2b22c2d8d39d None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:07:25.979428 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-732a6a0a-2266-4560-96f9-2b22c2d8d39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:26.077390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 738/1480] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:07:26.126073 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fe0f5-b4bc-418d-9da1-2743560f45e8 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:07:26.127851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7b9fe0f5-b4bc-418d-9da1-2743560f45e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:26.139123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:07:26.281764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fe0f5-b4bc-418d-9da1-2743560f45e8 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:07:26.366925 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:07:26.400086 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e32615-e198-4f2c-bb8d-436e114f4907 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:07:26.400735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e32615-e198-4f2c-bb8d-436e114f4907 service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:26.401231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e32615-e198-4f2c-bb8d-436e114f4907 service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:26.401625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e32615-e198-4f2c-bb8d-436e114f4907 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:26.403755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-69e32615-e198-4f2c-bb8d-436e114f4907 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:26.496445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e32615-e198-4f2c-bb8d-436e114f4907 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:07:26.498078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e32615-e198-4f2c-bb8d-436e114f4907 service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:07:26.516069 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 743/1481] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:07:28.784714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e7881-46a3-4a58-9c7c-2344fa926480 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:28.785324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e7881-46a3-4a58-9c7c-2344fa926480 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:28.785324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e7881-46a3-4a58-9c7c-2344fa926480 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:28.786684 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 739/1482] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:07:28.792205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f086153-bf22-443e-9a17-fbb11db17667 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:28.792663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f086153-bf22-443e-9a17-fbb11db17667 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:28.792778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f086153-bf22-443e-9a17-fbb11db17667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:28.862457 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-9f086153-bf22-443e-9a17-fbb11db17667 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:07:28.863728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-9f086153-bf22-443e-9a17-fbb11db17667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:28.910595 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:07:28.962886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 744/1483] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:07:29.001885 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e7881-46a3-4a58-9c7c-2344fa926480 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:07:29.003512 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e33e7881-46a3-4a58-9c7c-2344fa926480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:29.027128 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:07:29.108081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e7881-46a3-4a58-9c7c-2344fa926480 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:07:29.186688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a992051-7f3b-4e03-a4a5-bbad412b4496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:07:29.187376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a992051-7f3b-4e03-a4a5-bbad412b4496 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:29.187663 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a992051-7f3b-4e03-a4a5-bbad412b4496 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:29.187942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a992051-7f3b-4e03-a4a5-bbad412b4496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:29.190361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-8a992051-7f3b-4e03-a4a5-bbad412b4496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:29.269736 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:07:29.306375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a992051-7f3b-4e03-a4a5-bbad412b4496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:07:29.307851 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a992051-7f3b-4e03-a4a5-bbad412b4496 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:07:29.326133 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 740/1484] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:07:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:07:31.464545 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775f9d3-fcba-4832-8a13-23aab76c48f2 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:31.465184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775f9d3-fcba-4832-8a13-23aab76c48f2 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:31.465184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775f9d3-fcba-4832-8a13-23aab76c48f2 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:31.466506 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 745/1485] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:07:31.472322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc7ff0-0fb5-41b9-9f3a-d2f870e242ea None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:31.472835 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc7ff0-0fb5-41b9-9f3a-d2f870e242ea None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:31.472945 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc7ff0-0fb5-41b9-9f3a-d2f870e242ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:31.543718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-a4cc7ff0-0fb5-41b9-9f3a-d2f870e242ea None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:07:31.544963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a4cc7ff0-0fb5-41b9-9f3a-d2f870e242ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:31.645776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 741/1486] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:07:31.684884 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775f9d3-fcba-4832-8a13-23aab76c48f2 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:07:31.686439 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-9775f9d3-fcba-4832-8a13-23aab76c48f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:31.795981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775f9d3-fcba-4832-8a13-23aab76c48f2 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:07:31.869905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:07:31.871778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf903dff-b9d1-4ff3-9dc8-61f64cd64c99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:07:31.872444 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf903dff-b9d1-4ff3-9dc8-61f64cd64c99 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:31.872756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf903dff-b9d1-4ff3-9dc8-61f64cd64c99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:31.873032 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf903dff-b9d1-4ff3-9dc8-61f64cd64c99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:31.874731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-cf903dff-b9d1-4ff3-9dc8-61f64cd64c99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:31.986797 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf903dff-b9d1-4ff3-9dc8-61f64cd64c99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:07:31.988283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf903dff-b9d1-4ff3-9dc8-61f64cd64c99 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:07:32.006546 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 746/1487] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:07:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:07:34.390406 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017fa61-6a75-4372-8f9c-5bf3777741ea None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:34.391085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017fa61-6a75-4372-8f9c-5bf3777741ea None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:34.391085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017fa61-6a75-4372-8f9c-5bf3777741ea None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:34.392898 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 742/1488] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:07:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:07:34.399793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1551c4-34aa-4fb5-8644-55c8ef983e06 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:34.400314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1551c4-34aa-4fb5-8644-55c8ef983e06 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:34.400429 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1551c4-34aa-4fb5-8644-55c8ef983e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:34.480499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-1f1551c4-34aa-4fb5-8644-55c8ef983e06 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:07:34.481756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1f1551c4-34aa-4fb5-8644-55c8ef983e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:34.537816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:07:34.582971 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 747/1489] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:07:34.624198 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017fa61-6a75-4372-8f9c-5bf3777741ea None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:07:34.625755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d017fa61-6a75-4372-8f9c-5bf3777741ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:34.670643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:07:34.740741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017fa61-6a75-4372-8f9c-5bf3777741ea None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:07:34.816343 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47408183-6f10-466c-a7c9-a090e9cad8a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:07:34.816895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47408183-6f10-466c-a7c9-a090e9cad8a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:34.817184 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47408183-6f10-466c-a7c9-a090e9cad8a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:34.817463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47408183-6f10-466c-a7c9-a090e9cad8a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:34.819165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-47408183-6f10-466c-a7c9-a090e9cad8a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:34.901791 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:07:34.931439 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47408183-6f10-466c-a7c9-a090e9cad8a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:07:34.932897 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47408183-6f10-466c-a7c9-a090e9cad8a2 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:07:34.951649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 743/1490] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:07:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:07:37.495664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6d465-dda3-472f-8a14-3d170a5ed2c3 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:37.496303 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6d465-dda3-472f-8a14-3d170a5ed2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:37.496303 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6d465-dda3-472f-8a14-3d170a5ed2c3 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:37.497645 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 748/1491] 10.222.0.83 () {54 vars in 926 bytes} [Mon Jul 18 15:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:07:37.504027 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ccf9c-9795-4798-b01e-500fa452b0c3 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:37.504530 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ccf9c-9795-4798-b01e-500fa452b0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:37.504643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ccf9c-9795-4798-b01e-500fa452b0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:37.573892 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-9f7ccf9c-9795-4798-b01e-500fa452b0c3 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:07:37.575157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9f7ccf9c-9795-4798-b01e-500fa452b0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:37.674782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 744/1492] 10.222.0.83 () {58 vars in 1023 bytes} [Mon Jul 18 15:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:07:37.714733 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6d465-dda3-472f-8a14-3d170a5ed2c3 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:07:37.716291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4ba6d465-dda3-472f-8a14-3d170a5ed2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:37.822257 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6d465-dda3-472f-8a14-3d170a5ed2c3 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:07:37.895621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b42c7-4f29-446d-9319-a0663419066a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:07:37.896278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b42c7-4f29-446d-9319-a0663419066a service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:07:37.896567 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b42c7-4f29-446d-9319-a0663419066a service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:07:37.896849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b42c7-4f29-446d-9319-a0663419066a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:07:37.898518 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d36b42c7-4f29-446d-9319-a0663419066a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:07:37.907886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:07:38.013020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b42c7-4f29-446d-9319-a0663419066a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:07:38.014484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b42c7-4f29-446d-9319-a0663419066a service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:07:38.034327 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 749/1493] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:07:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:07:47.672230 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=82 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:07:48.030952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=50 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:09:16.671577 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6dab8-ab39-4b7e-842e-7fb330ec395e None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:16.673816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6dab8-ab39-4b7e-842e-7fb330ec395e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:16.673816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6dab8-ab39-4b7e-842e-7fb330ec395e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:16.748789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-26e6dab8-ab39-4b7e-842e-7fb330ec395e None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:09:16.750031 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-26e6dab8-ab39-4b7e-842e-7fb330ec395e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:09:16.852584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 745/1494] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:09:16.861139 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b100b5-4978-414b-8a91-195065882c77 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:16.861665 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b100b5-4978-414b-8a91-195065882c77 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:16.861780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b100b5-4978-414b-8a91-195065882c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:16.939780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-37b100b5-4978-414b-8a91-195065882c77 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:09:16.941009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-37b100b5-4978-414b-8a91-195065882c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:09:17.044279 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 750/1495] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:09:17.049082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6dab8-ab39-4b7e-842e-7fb330ec395e None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:09:17.050292 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-26e6dab8-ab39-4b7e-842e-7fb330ec395e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:09:17.062456 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6dab8-ab39-4b7e-842e-7fb330ec395e None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:09:17.125231 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:09:17.159837 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6068fa-53b4-4fb4-9236-4cdcbf4039b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:09:17.160506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6068fa-53b4-4fb4-9236-4cdcbf4039b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:17.160790 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6068fa-53b4-4fb4-9236-4cdcbf4039b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:17.161069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6068fa-53b4-4fb4-9236-4cdcbf4039b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:17.161811 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6068fa-53b4-4fb4-9236-4cdcbf4039b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:09:17.162119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6068fa-53b4-4fb4-9236-4cdcbf4039b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:09:17.163177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: WARNING py.warnings [None req-ac6068fa-53b4-4fb4-9236-4cdcbf4039b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 15:09:17.163177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: warnings.warn(msg) Jul 18 15:09:17.163177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: Jul 18 15:09:17.164067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6068fa-53b4-4fb4-9236-4cdcbf4039b1 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:09:17.176604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 746/1496] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:09:17.185986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b100b5-4978-414b-8a91-195065882c77 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:09:17.195089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b100b5-4978-414b-8a91-195065882c77 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:09:17.380818 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:09:17.415541 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39af04-a175-4e32-8b22-f08c42518e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:09:17.416205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39af04-a175-4e32-8b22-f08c42518e9d admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:17.416489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39af04-a175-4e32-8b22-f08c42518e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:17.416764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39af04-a175-4e32-8b22-f08c42518e9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:17.418175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39af04-a175-4e32-8b22-f08c42518e9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:09:17.420072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: WARNING py.warnings [None req-8c39af04-a175-4e32-8b22-f08c42518e9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 15:09:17.420072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: warnings.warn(msg) Jul 18 15:09:17.420072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: Jul 18 15:09:17.421285 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39af04-a175-4e32-8b22-f08c42518e9d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:09:17.541031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 751/1497] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:09:17 2022] POST /identity/v3/projects => generated 377 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:09:17.545974 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6068fa-53b4-4fb4-9236-4cdcbf4039b1 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:09:17.547536 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ac6068fa-53b4-4fb4-9236-4cdcbf4039b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:09:17.581081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6068fa-53b4-4fb4-9236-4cdcbf4039b1 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:09:17.676308 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453fd57-dfc9-494e-b678-53cdf37992f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:09:17.677182 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453fd57-dfc9-494e-b678-53cdf37992f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:17.677458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453fd57-dfc9-494e-b678-53cdf37992f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:17.677817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453fd57-dfc9-494e-b678-53cdf37992f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:17.678587 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453fd57-dfc9-494e-b678-53cdf37992f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:09:17.679832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: WARNING py.warnings [None req-b453fd57-dfc9-494e-b678-53cdf37992f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 15:09:17.679832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: warnings.warn(msg) Jul 18 15:09:17.679832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: Jul 18 15:09:17.681056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453fd57-dfc9-494e-b678-53cdf37992f4 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:09:17.826349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 747/1498] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:09:17 2022] POST /identity/v3/users => generated 289 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:09:17.830681 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39af04-a175-4e32-8b22-f08c42518e9d admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:09:17.832190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-8c39af04-a175-4e32-8b22-f08c42518e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:09:17.895045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39af04-a175-4e32-8b22-f08c42518e9d admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:09:17.982199 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:09:17.992818 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e0c81-ea18-48e3-bc62-c65871d4e4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:09:17.996055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e0c81-ea18-48e3-bc62-c65871d4e4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:17.996055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e0c81-ea18-48e3-bc62-c65871d4e4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:17.996055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e0c81-ea18-48e3-bc62-c65871d4e4c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:17.996055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e0c81-ea18-48e3-bc62-c65871d4e4c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:09:17.996462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: WARNING py.warnings [None req-4e0e0c81-ea18-48e3-bc62-c65871d4e4c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 15:09:17.996462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: warnings.warn(msg) Jul 18 15:09:17.996462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: Jul 18 15:09:17.997319 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e0c81-ea18-48e3-bc62-c65871d4e4c8 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:09:18.010303 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 752/1499] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:09:18.015072 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453fd57-dfc9-494e-b678-53cdf37992f4 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:09:18.016624 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b453fd57-dfc9-494e-b678-53cdf37992f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:09:18.029135 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453fd57-dfc9-494e-b678-53cdf37992f4 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:09:18.055886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:09:18.129643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc73d54-5b4e-4c42-8dcc-cd4193d199ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:09:18.130218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc73d54-5b4e-4c42-8dcc-cd4193d199ff admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:18.130503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc73d54-5b4e-4c42-8dcc-cd4193d199ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:18.130824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc73d54-5b4e-4c42-8dcc-cd4193d199ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:18.131551 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc73d54-5b4e-4c42-8dcc-cd4193d199ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:09:18.132576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: WARNING py.warnings [None req-3cc73d54-5b4e-4c42-8dcc-cd4193d199ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 15:09:18.132576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: warnings.warn(msg) Jul 18 15:09:18.132576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: Jul 18 15:09:18.133475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc73d54-5b4e-4c42-8dcc-cd4193d199ff admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:09:18.146357 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 748/1500] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:09:18.150940 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e0c81-ea18-48e3-bc62-c65871d4e4c8 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:09:18.153864 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4e0e0c81-ea18-48e3-bc62-c65871d4e4c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:09:18.187350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e0c81-ea18-48e3-bc62-c65871d4e4c8 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:09:18.283210 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb39c5b-ccbd-41a7-b3b0-c6f320560833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:09:18.284103 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb39c5b-ccbd-41a7-b3b0-c6f320560833 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:18.284389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb39c5b-ccbd-41a7-b3b0-c6f320560833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:18.284666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb39c5b-ccbd-41a7-b3b0-c6f320560833 admin admin] PATH_INFO: `/v3/projects/e715358e0d514abdb48af51e0cd676d6/users/d4011e25e4804146a3a8aa359662b9a6/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:18.308570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb39c5b-ccbd-41a7-b3b0-c6f320560833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e715358e0d514abdb48af51e0cd676d6, user_id=d4011e25e4804146a3a8aa359662b9a6, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:09:18.309619 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: WARNING py.warnings [None req-7eb39c5b-ccbd-41a7-b3b0-c6f320560833 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 15:09:18.309619 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: warnings.warn(msg) Jul 18 15:09:18.309619 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: Jul 18 15:09:18.311605 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb39c5b-ccbd-41a7-b3b0-c6f320560833 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:09:18.452134 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 753/1501] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:09:18 2022] PUT /identity/v3/projects/e715358e0d514abdb48af51e0cd676d6/users/d4011e25e4804146a3a8aa359662b9a6/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:09:18.464624 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42b590-efd5-4ca5-9899-4364ba13823a None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:18.464940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42b590-efd5-4ca5-9899-4364ba13823a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:18.465209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42b590-efd5-4ca5-9899-4364ba13823a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:18.522386 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-4d42b590-efd5-4ca5-9899-4364ba13823a None None] MFA Rules not processed for user `d4011e25e4804146a3a8aa359662b9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:09:18.523665 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-4d42b590-efd5-4ca5-9899-4364ba13823a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:09:18.603448 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 749/1502] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:09:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:09:18.617331 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cd7a4-9fde-4916-8f5c-eef1c25f106d None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:18.617672 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cd7a4-9fde-4916-8f5c-eef1c25f106d None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:18.617950 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cd7a4-9fde-4916-8f5c-eef1c25f106d None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:18.619371 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 754/1503] 10.222.0.83 () {54 vars in 964 bytes} [Mon Jul 18 15:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:09:18.624972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a50419-7c27-42fe-9560-121f1bdea816 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:18.625280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a50419-7c27-42fe-9560-121f1bdea816 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:18.625850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a50419-7c27-42fe-9560-121f1bdea816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:18.657440 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:09:18.703418 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b4a50419-7c27-42fe-9560-121f1bdea816 None None] MFA Rules not processed for user `9218503bc8a647d68ffab6945bae456a`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:09:18.704575 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b4a50419-7c27-42fe-9560-121f1bdea816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:09:18.782164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 750/1504] 10.222.0.83 () {58 vars in 1061 bytes} [Mon Jul 18 15:09:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:09:18.789928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cd7a4-9fde-4916-8f5c-eef1c25f106d None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:09:18.791481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-601cd7a4-9fde-4916-8f5c-eef1c25f106d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:09:18.844615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:09:18.899311 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cd7a4-9fde-4916-8f5c-eef1c25f106d None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:09:18.975237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c4d80-05e2-4af7-9db5-377030a5c165 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:09:18.975790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c4d80-05e2-4af7-9db5-377030a5c165 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:18.976072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c4d80-05e2-4af7-9db5-377030a5c165 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:18.976346 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c4d80-05e2-4af7-9db5-377030a5c165 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:18.978025 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c58c4d80-05e2-4af7-9db5-377030a5c165 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:09:19.082894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c4d80-05e2-4af7-9db5-377030a5c165 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:09:19.084370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c4d80-05e2-4af7-9db5-377030a5c165 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:09:19.087920 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:09:19.103107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 755/1505] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:09:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:09:19.146478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f82be-6123-4268-b8ee-bd4b89ad7d11 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:19.146809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f82be-6123-4268-b8ee-bd4b89ad7d11 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:19.147093 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f82be-6123-4268-b8ee-bd4b89ad7d11 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:19.148409 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 751/1506] 10.222.0.83 () {52 vars in 829 bytes} [Mon Jul 18 15:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:09:19.152932 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c4d80-05e2-4af7-9db5-377030a5c165 service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:09:19.154207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c58c4d80-05e2-4af7-9db5-377030a5c165 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:09:19.256430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c4d80-05e2-4af7-9db5-377030a5c165 service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:09:19.352640 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966121c1-5e45-4777-9f51-030cb3f68b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:09:19.353527 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966121c1-5e45-4777-9f51-030cb3f68b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:19.353813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966121c1-5e45-4777-9f51-030cb3f68b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:19.354100 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966121c1-5e45-4777-9f51-030cb3f68b17 admin admin] PATH_INFO: `/v3/users/d4011e25e4804146a3a8aa359662b9a6` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:19.368300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966121c1-5e45-4777-9f51-030cb3f68b17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4011e25e4804146a3a8aa359662b9a6)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:09:19.369344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: WARNING py.warnings [None req-966121c1-5e45-4777-9f51-030cb3f68b17 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 15:09:19.369344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: warnings.warn(msg) Jul 18 15:09:19.369344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: Jul 18 15:09:19.371752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966121c1-5e45-4777-9f51-030cb3f68b17 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:09:19.382569 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:09:19.828089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-966121c1-5e45-4777-9f51-030cb3f68b17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4011e25e4804146a3a8aa359662b9a6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:09:19.841079 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-966121c1-5e45-4777-9f51-030cb3f68b17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4011e25e4804146a3a8aa359662b9a6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:09:19.841674 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-966121c1-5e45-4777-9f51-030cb3f68b17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4011e25e4804146a3a8aa359662b9a6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:09:19.861018 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-966121c1-5e45-4777-9f51-030cb3f68b17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4011e25e4804146a3a8aa359662b9a6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:09:19.921426 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-966121c1-5e45-4777-9f51-030cb3f68b17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4011e25e4804146a3a8aa359662b9a6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:09:19.927534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 756/1507] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:09:19 2022] DELETE /identity/v3/users/d4011e25e4804146a3a8aa359662b9a6 => generated 0 bytes in 776 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:09:19.931961 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbd370-7a26-4c4e-9aba-b776398e2f50 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:19.932422 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbd370-7a26-4c4e-9aba-b776398e2f50 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:19.932578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbd370-7a26-4c4e-9aba-b776398e2f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:19.992366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-4bcbd370-7a26-4c4e-9aba-b776398e2f50 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:09:19.993582 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-4bcbd370-7a26-4c4e-9aba-b776398e2f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:09:20.069302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:09:20.082293 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 752/1508] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:09:20.092860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966121c1-5e45-4777-9f51-030cb3f68b17 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:09:20.094404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-966121c1-5e45-4777-9f51-030cb3f68b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:09:20.195258 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966121c1-5e45-4777-9f51-030cb3f68b17 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:09:20.251909 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:09:20.274773 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561781a-23b1-4e3a-8e65-15f705137398 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:09:20.275446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561781a-23b1-4e3a-8e65-15f705137398 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:20.275738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561781a-23b1-4e3a-8e65-15f705137398 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:20.276029 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561781a-23b1-4e3a-8e65-15f705137398 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:20.278348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-9561781a-23b1-4e3a-8e65-15f705137398 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:09:20.392559 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561781a-23b1-4e3a-8e65-15f705137398 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:09:20.393956 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561781a-23b1-4e3a-8e65-15f705137398 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:09:20.415699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 757/1509] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:09:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:09:20.941829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbd370-7a26-4c4e-9aba-b776398e2f50 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:09:20.943133 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-4bcbd370-7a26-4c4e-9aba-b776398e2f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:09:20.956783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbd370-7a26-4c4e-9aba-b776398e2f50 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:09:21.061652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8762a-0a39-42bb-ae40-2e9a6fbefc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:09:21.063201 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8762a-0a39-42bb-ae40-2e9a6fbefc18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:09:21.063707 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8762a-0a39-42bb-ae40-2e9a6fbefc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:09:21.063996 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8762a-0a39-42bb-ae40-2e9a6fbefc18 admin admin] PATH_INFO: `/v3/projects/e715358e0d514abdb48af51e0cd676d6` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:09:21.075122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8762a-0a39-42bb-ae40-2e9a6fbefc18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e715358e0d514abdb48af51e0cd676d6)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:09:21.076166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: WARNING py.warnings [None req-cae8762a-0a39-42bb-ae40-2e9a6fbefc18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 15:09:21.076166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: warnings.warn(msg) Jul 18 15:09:21.076166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: Jul 18 15:09:21.077356 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8762a-0a39-42bb-ae40-2e9a6fbefc18 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:09:21.195534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cae8762a-0a39-42bb-ae40-2e9a6fbefc18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e715358e0d514abdb48af51e0cd676d6', 'request_id': 'req-cae8762a-0a39-42bb-ae40-2e9a6fbefc18'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:09:21.209750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cae8762a-0a39-42bb-ae40-2e9a6fbefc18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e715358e0d514abdb48af51e0cd676d6', 'request_id': 'req-cae8762a-0a39-42bb-ae40-2e9a6fbefc18'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:09:21.216470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cae8762a-0a39-42bb-ae40-2e9a6fbefc18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e715358e0d514abdb48af51e0cd676d6', 'request_id': 'req-cae8762a-0a39-42bb-ae40-2e9a6fbefc18'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:09:21.316690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cae8762a-0a39-42bb-ae40-2e9a6fbefc18 admin admin] The token cache is being invalidate because project e715358e0d514abdb48af51e0cd676d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:09:21.316971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cae8762a-0a39-42bb-ae40-2e9a6fbefc18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:09:21.317398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cae8762a-0a39-42bb-ae40-2e9a6fbefc18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:09:21.318916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 753/1510] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:09:20 2022] DELETE /identity/v3/projects/e715358e0d514abdb48af51e0cd676d6 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:09:30.413350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=62 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:09:31.314611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:11:07.061131 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c59520-90de-470f-b4bf-7e09c7618abb None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:07.061807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c59520-90de-470f-b4bf-7e09c7618abb None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:07.061807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c59520-90de-470f-b4bf-7e09c7618abb None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:07.063559 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 15:11:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 15:11:07.070963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c596265-357c-4909-ba93-85d88fffb891 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:07.071613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c596265-357c-4909-ba93-85d88fffb891 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:07.071613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c596265-357c-4909-ba93-85d88fffb891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:07.213797 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-0c596265-357c-4909-ba93-85d88fffb891 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:11:07.215295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0c596265-357c-4909-ba93-85d88fffb891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:11:07.360050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 15:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:11:07.536131 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c59520-90de-470f-b4bf-7e09c7618abb None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:11:07.539679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-74c59520-90de-470f-b4bf-7e09c7618abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:11:07.715304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c59520-90de-470f-b4bf-7e09c7618abb None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:11:07.723223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4a341-05c6-4f92-a765-b9d7582289a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:11:07.724324 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4a341-05c6-4f92-a765-b9d7582289a8 service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:07.724814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4a341-05c6-4f92-a765-b9d7582289a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:07.725291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4a341-05c6-4f92-a765-b9d7582289a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:07.741172 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4a341-05c6-4f92-a765-b9d7582289a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:11:07.744209 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4a341-05c6-4f92-a765-b9d7582289a8 service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:11:07.764231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 759/1513] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:11:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:11:08.745674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db3fac-630f-40ef-bc6d-d3fd678f2728 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:08.746313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db3fac-630f-40ef-bc6d-d3fd678f2728 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:08.746829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db3fac-630f-40ef-bc6d-d3fd678f2728 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:08.749614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 15:11:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:11:08.755704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d3ede-0693-4cde-b033-0283e0b51d52 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:08.756440 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d3ede-0693-4cde-b033-0283e0b51d52 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:08.756440 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d3ede-0693-4cde-b033-0283e0b51d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:08.775693 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bbbd5-77c8-4917-9e5f-a7688e3206a5 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:08.776145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bbbd5-77c8-4917-9e5f-a7688e3206a5 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:08.776590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bbbd5-77c8-4917-9e5f-a7688e3206a5 None None] PATH_INFO: `/v3` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:08.777644 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-355d3ede-0693-4cde-b033-0283e0b51d52 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:11:08.778278 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 756/1515] 10.222.0.91 () {54 vars in 925 bytes} [Mon Jul 18 15:11:08 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 15:11:08.779065 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-355d3ede-0693-4cde-b033-0283e0b51d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:11:08.786514 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be35e3-92bf-4b06-a5cf-6c78c74e959f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:08.790158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be35e3-92bf-4b06-a5cf-6c78c74e959f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:08.790158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be35e3-92bf-4b06-a5cf-6c78c74e959f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:08.790528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 760/1516] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 15:11:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:11:08.891248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-86be35e3-92bf-4b06-a5cf-6c78c74e959f None None] MFA Rules not processed for user `76cc0ae48b5a49c6a7a1c11a08c389e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:11:08.892719 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-86be35e3-92bf-4b06-a5cf-6c78c74e959f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:11:08.955057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:11:08.957452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d3ede-0693-4cde-b033-0283e0b51d52 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:11:08.977978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d3ede-0693-4cde-b033-0283e0b51d52 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:11:08.991401 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166a8b3-2942-471b-88dd-f11c49d3acb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:11:08.992459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166a8b3-2942-471b-88dd-f11c49d3acb6 service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:08.992926 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166a8b3-2942-471b-88dd-f11c49d3acb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:08.993353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166a8b3-2942-471b-88dd-f11c49d3acb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:09.010170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b166a8b3-2942-471b-88dd-f11c49d3acb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:11:09.013194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b166a8b3-2942-471b-88dd-f11c49d3acb6 service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:11:09.032227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 761/1517] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:11:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:11:09.037485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 757/1518] 10.222.0.91 () {58 vars in 1015 bytes} [Mon Jul 18 15:11:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:11:13.305579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e340e2a-8933-40a7-9d97-d40025d9316e None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:13.306387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e340e2a-8933-40a7-9d97-d40025d9316e None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:13.306610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e340e2a-8933-40a7-9d97-d40025d9316e None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:13.308288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 762/1519] 10.222.0.83 () {54 vars in 961 bytes} [Mon Jul 18 15:11:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:11:13.314368 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63a75e-f6d1-4359-a959-092730d76f0c None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:13.315698 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63a75e-f6d1-4359-a959-092730d76f0c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:13.315910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63a75e-f6d1-4359-a959-092730d76f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:13.335511 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-7e63a75e-f6d1-4359-a959-092730d76f0c None None] MFA Rules not processed for user `76cc0ae48b5a49c6a7a1c11a08c389e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:11:13.339264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7e63a75e-f6d1-4359-a959-092730d76f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:11:13.346348 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 758/1520] 10.222.0.83 () {58 vars in 1058 bytes} [Mon Jul 18 15:11:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:11:13.354944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e340e2a-8933-40a7-9d97-d40025d9316e None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:11:13.371510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e340e2a-8933-40a7-9d97-d40025d9316e None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:11:13.377862 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f43c4-59c6-40b0-8917-9b0b88114fe4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76cc0ae48b5a49c6a7a1c11a08c389e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:11:13.378771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f43c4-59c6-40b0-8917-9b0b88114fe4 service placement] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:13.379074 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f43c4-59c6-40b0-8917-9b0b88114fe4 service placement] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:13.379356 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f43c4-59c6-40b0-8917-9b0b88114fe4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:13.389286 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f43c4-59c6-40b0-8917-9b0b88114fe4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:11:13.390990 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f43c4-59c6-40b0-8917-9b0b88114fe4 service placement] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:11:13.403245 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 763/1521] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:11:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:11:20.534360 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8f64c-45d2-4b04-bcfc-caf3ade3e62a None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:20.535062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8f64c-45d2-4b04-bcfc-caf3ade3e62a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:20.535062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8f64c-45d2-4b04-bcfc-caf3ade3e62a None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:20.537069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 15:11:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:11:20.543318 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595ae01-aa32-4568-8165-c9520c7ce85b None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:20.544193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595ae01-aa32-4568-8165-c9520c7ce85b None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:20.544193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595ae01-aa32-4568-8165-c9520c7ce85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:20.565807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-5595ae01-aa32-4568-8165-c9520c7ce85b None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:11:20.567463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-5595ae01-aa32-4568-8165-c9520c7ce85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:11:20.576419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 15:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:11:20.747862 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8f64c-45d2-4b04-bcfc-caf3ade3e62a None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:11:20.764254 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8f64c-45d2-4b04-bcfc-caf3ade3e62a None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:11:20.770834 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc535c-b286-4827-beb3-2439d1ab307b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:11:20.771673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc535c-b286-4827-beb3-2439d1ab307b service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:20.772079 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc535c-b286-4827-beb3-2439d1ab307b service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:20.772468 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc535c-b286-4827-beb3-2439d1ab307b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:20.782877 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc535c-b286-4827-beb3-2439d1ab307b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:11:20.785331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc535c-b286-4827-beb3-2439d1ab307b service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:11:20.798016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 760/1524] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:11:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:11:21.741355 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612e29f-b594-426f-8e5d-d3423191a407 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:21.742101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612e29f-b594-426f-8e5d-d3423191a407 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:21.742101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612e29f-b594-426f-8e5d-d3423191a407 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:21.743420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 15:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:11:21.749535 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc9379-c42c-4127-8aab-f79424ae22ce None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:21.750136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc9379-c42c-4127-8aab-f79424ae22ce None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:21.750136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc9379-c42c-4127-8aab-f79424ae22ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:21.771712 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-2fdc9379-c42c-4127-8aab-f79424ae22ce None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:11:21.773326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2fdc9379-c42c-4127-8aab-f79424ae22ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:11:21.783399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 15:11:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:11:21.951755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612e29f-b594-426f-8e5d-d3423191a407 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:11:21.969737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612e29f-b594-426f-8e5d-d3423191a407 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:11:21.976854 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3eea2a-3f50-47ed-92c1-002ed178e8e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:11:21.977656 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3eea2a-3f50-47ed-92c1-002ed178e8e2 service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:21.977856 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3eea2a-3f50-47ed-92c1-002ed178e8e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:21.978109 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3eea2a-3f50-47ed-92c1-002ed178e8e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:21.988417 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3eea2a-3f50-47ed-92c1-002ed178e8e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:11:21.990185 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3eea2a-3f50-47ed-92c1-002ed178e8e2 service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:11:22.002374 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 766/1527] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:11:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:11:26.353449 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df1fca-3255-4024-a66c-75bd73025702 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:26.354188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df1fca-3255-4024-a66c-75bd73025702 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:26.354188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df1fca-3255-4024-a66c-75bd73025702 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:26.355852 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 15:11:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:11:26.365649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e84bc-d0a1-4339-ac5e-36dc011f8c0e None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:26.366614 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e84bc-d0a1-4339-ac5e-36dc011f8c0e None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:26.366614 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e84bc-d0a1-4339-ac5e-36dc011f8c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:26.387289 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-321e84bc-d0a1-4339-ac5e-36dc011f8c0e None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:11:26.388580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-321e84bc-d0a1-4339-ac5e-36dc011f8c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:11:26.397370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 15:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:11:26.440542 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df1fca-3255-4024-a66c-75bd73025702 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:11:26.442517 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-23df1fca-3255-4024-a66c-75bd73025702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:11:26.540571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df1fca-3255-4024-a66c-75bd73025702 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:11:26.545236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea371bf8-1826-47b6-8f46-210033a8c871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:11:26.545909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea371bf8-1826-47b6-8f46-210033a8c871 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:26.546198 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea371bf8-1826-47b6-8f46-210033a8c871 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:26.546478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea371bf8-1826-47b6-8f46-210033a8c871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:26.558024 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea371bf8-1826-47b6-8f46-210033a8c871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:11:26.559585 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea371bf8-1826-47b6-8f46-210033a8c871 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:11:26.571178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 763/1530] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:11:28.769104 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e41c8-98df-4f71-8893-764d669279fe None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:28.769830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e41c8-98df-4f71-8893-764d669279fe None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:28.769830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e41c8-98df-4f71-8893-764d669279fe None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:28.771432 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 15:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:11:28.778059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f9d1c-1360-43c2-9ed1-45c710c20a87 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:28.778744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f9d1c-1360-43c2-9ed1-45c710c20a87 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:28.778744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f9d1c-1360-43c2-9ed1-45c710c20a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:28.800332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-451f9d1c-1360-43c2-9ed1-45c710c20a87 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:11:28.801739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-451f9d1c-1360-43c2-9ed1-45c710c20a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:11:28.810353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 15:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:11:28.850001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e41c8-98df-4f71-8893-764d669279fe None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:11:28.862130 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e41c8-98df-4f71-8893-764d669279fe None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:11:28.865922 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c373ef1-096a-4f38-bce5-b0e6f08b9846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:11:28.866614 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c373ef1-096a-4f38-bce5-b0e6f08b9846 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:28.866902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c373ef1-096a-4f38-bce5-b0e6f08b9846 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:28.867227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c373ef1-096a-4f38-bce5-b0e6f08b9846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:28.880574 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c373ef1-096a-4f38-bce5-b0e6f08b9846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:11:28.882086 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c373ef1-096a-4f38-bce5-b0e6f08b9846 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:11:28.894516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 769/1533] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:11:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:11:32.927462 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a91238-e8bb-4c27-aef5-e03e00db13ce None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:32.928235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a91238-e8bb-4c27-aef5-e03e00db13ce None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:32.928235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a91238-e8bb-4c27-aef5-e03e00db13ce None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:32.929421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 15:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:11:32.937872 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d77acd-e04c-413c-8689-b96035b92cf6 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:32.938659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d77acd-e04c-413c-8689-b96035b92cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:32.939291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d77acd-e04c-413c-8689-b96035b92cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:32.945814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=87 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:11:32.959237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-12d77acd-e04c-413c-8689-b96035b92cf6 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:11:32.960484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-12d77acd-e04c-413c-8689-b96035b92cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:11:32.968843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 15:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:11:33.311883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a91238-e8bb-4c27-aef5-e03e00db13ce None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:11:33.313668 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-60a91238-e8bb-4c27-aef5-e03e00db13ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:11:33.403866 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a91238-e8bb-4c27-aef5-e03e00db13ce None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:11:33.408617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068cc85-44ba-4749-a9a7-1a96ec6c4d4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:11:33.409288 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068cc85-44ba-4749-a9a7-1a96ec6c4d4a service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:33.409644 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068cc85-44ba-4749-a9a7-1a96ec6c4d4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:33.409929 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068cc85-44ba-4749-a9a7-1a96ec6c4d4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:33.418501 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:11:33.421765 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7068cc85-44ba-4749-a9a7-1a96ec6c4d4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:11:33.423577 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7068cc85-44ba-4749-a9a7-1a96ec6c4d4a service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:11:33.434925 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 766/1536] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:11:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:11:35.706431 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e77ad0-c6ad-479b-87ee-7044d67389ce None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:35.707297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e77ad0-c6ad-479b-87ee-7044d67389ce None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:35.707297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e77ad0-c6ad-479b-87ee-7044d67389ce None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:35.708748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 15:11:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 15:11:35.715167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b678861-8655-466d-884f-2dd4d0928909 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:35.716114 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b678861-8655-466d-884f-2dd4d0928909 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:35.716114 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b678861-8655-466d-884f-2dd4d0928909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:35.738138 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-6b678861-8655-466d-884f-2dd4d0928909 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:11:35.747926 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6b678861-8655-466d-884f-2dd4d0928909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:11:35.748185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 15:11:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:11:36.080368 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e77ad0-c6ad-479b-87ee-7044d67389ce None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:11:36.082313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-76e77ad0-c6ad-479b-87ee-7044d67389ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:11:36.096525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e77ad0-c6ad-479b-87ee-7044d67389ce None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:11:36.100184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18952e56-a4c6-4e33-9790-9bcfac723679 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:11:36.100854 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18952e56-a4c6-4e33-9790-9bcfac723679 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:11:36.101137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18952e56-a4c6-4e33-9790-9bcfac723679 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:11:36.101414 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18952e56-a4c6-4e33-9790-9bcfac723679 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:11:36.112606 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18952e56-a4c6-4e33-9790-9bcfac723679 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:11:36.114097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18952e56-a4c6-4e33-9790-9bcfac723679 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:11:36.124808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 772/1539] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:11:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:11:45.732369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=5 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:11:46.120107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=8 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:11.695686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9c249-4375-4a59-9667-7a737f8477a2 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:11.696884 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9c249-4375-4a59-9667-7a737f8477a2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:11.697203 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9c249-4375-4a59-9667-7a737f8477a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:11.733687 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f66a94-c0e1-49b0-91c0-4fdf4670a254 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:11.735517 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f66a94-c0e1-49b0-91c0-4fdf4670a254 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:11.736352 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f66a94-c0e1-49b0-91c0-4fdf4670a254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:11.740739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-77e9c249-4375-4a59-9667-7a737f8477a2 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:11.741364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-77e9c249-4375-4a59-9667-7a737f8477a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:11.783064 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 768/1540] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:11.784268 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-22f66a94-c0e1-49b0-91c0-4fdf4670a254 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:11.790580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb94c49-927b-4834-b181-6bcf93f8c81a None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:11.790580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb94c49-927b-4834-b181-6bcf93f8c81a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:11.790580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb94c49-927b-4834-b181-6bcf93f8c81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:11.791860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-22f66a94-c0e1-49b0-91c0-4fdf4670a254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:11.833874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 773/1541] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:13:11.842828 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-7bb94c49-927b-4834-b181-6bcf93f8c81a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:11.845186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7bb94c49-927b-4834-b181-6bcf93f8c81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:11.856605 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6fb7d-19e2-433a-a11b-04b815f53ee7 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:11.861459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6fb7d-19e2-433a-a11b-04b815f53ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:11.864451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6fb7d-19e2-433a-a11b-04b815f53ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:11.875228 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 769/1542] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:11.877060 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb94c49-927b-4834-b181-6bcf93f8c81a None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:11.907207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb94c49-927b-4834-b181-6bcf93f8c81a None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:11.931823 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-20d6fb7d-19e2-433a-a11b-04b815f53ee7 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:11.933535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-20d6fb7d-19e2-433a-a11b-04b815f53ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:11.942590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ce9a3-117b-45a8-a6d3-7d97483e8ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:11.944410 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ce9a3-117b-45a8-a6d3-7d97483e8ddb admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:11.945194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ce9a3-117b-45a8-a6d3-7d97483e8ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:11.946694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ce9a3-117b-45a8-a6d3-7d97483e8ddb admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:11.954386 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ce9a3-117b-45a8-a6d3-7d97483e8ddb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:13:11.954977 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 774/1543] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:13:11.956031 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ce9a3-117b-45a8-a6d3-7d97483e8ddb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:11.956631 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d8f42-d500-4cab-a8a9-20650506bf15 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:11.957006 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d8f42-d500-4cab-a8a9-20650506bf15 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:11.957292 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d8f42-d500-4cab-a8a9-20650506bf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:11.960873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ce9a3-117b-45a8-a6d3-7d97483e8ddb admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:12.001156 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 770/1544] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 15:13:12.006659 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ce9a3-117b-45a8-a6d3-7d97483e8ddb admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:12.009630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-831d8f42-d500-4cab-a8a9-20650506bf15 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:12.011756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-831d8f42-d500-4cab-a8a9-20650506bf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:12.037287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ce9a3-117b-45a8-a6d3-7d97483e8ddb admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:12.043847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 775/1545] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:12.047533 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d8f42-d500-4cab-a8a9-20650506bf15 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:12.068896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbf67e-1d76-4b64-a171-ce0dbafd8ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:12.070768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbf67e-1d76-4b64-a171-ce0dbafd8ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.071682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbf67e-1d76-4b64-a171-ce0dbafd8ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.072375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbf67e-1d76-4b64-a171-ce0dbafd8ee5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.073796 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbf67e-1d76-4b64-a171-ce0dbafd8ee5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:13:12.074390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbf67e-1d76-4b64-a171-ce0dbafd8ee5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:12.081858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbf67e-1d76-4b64-a171-ce0dbafd8ee5 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:12.082208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d8f42-d500-4cab-a8a9-20650506bf15 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:12.101524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee4a47-0132-4754-9afb-d64c189547e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:12.103417 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee4a47-0132-4754-9afb-d64c189547e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.103911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee4a47-0132-4754-9afb-d64c189547e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.104420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee4a47-0132-4754-9afb-d64c189547e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.105635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 771/1546] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:13:12.111135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee4a47-0132-4754-9afb-d64c189547e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:12.111635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781ef04-2f9f-4075-80c3-e6fd4871c30f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.112600 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781ef04-2f9f-4075-80c3-e6fd4871c30f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.112840 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781ef04-2f9f-4075-80c3-e6fd4871c30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.115461 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee4a47-0132-4754-9afb-d64c189547e9 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:12.140871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-0781ef04-2f9f-4075-80c3-e6fd4871c30f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:12.142626 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0781ef04-2f9f-4075-80c3-e6fd4871c30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:12.169472 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 772/1547] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:12.176336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd57f2a-ffde-4916-9edd-03fc5f05bfa7 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.176851 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd57f2a-ffde-4916-9edd-03fc5f05bfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.177139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd57f2a-ffde-4916-9edd-03fc5f05bfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.223248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-ffd57f2a-ffde-4916-9edd-03fc5f05bfa7 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:12.223248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ffd57f2a-ffde-4916-9edd-03fc5f05bfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:12.235246 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 773/1548] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:12.237549 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd57f2a-ffde-4916-9edd-03fc5f05bfa7 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:12.258778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd57f2a-ffde-4916-9edd-03fc5f05bfa7 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:12.267450 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f402b20-d5c8-4353-adea-c6036e54012d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:12.268538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f402b20-d5c8-4353-adea-c6036e54012d admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.269047 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f402b20-d5c8-4353-adea-c6036e54012d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.270328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f402b20-d5c8-4353-adea-c6036e54012d admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.271002 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f402b20-d5c8-4353-adea-c6036e54012d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:12.273111 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: WARNING py.warnings [None req-0f402b20-d5c8-4353-adea-c6036e54012d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 15:13:12.273111 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: warnings.warn(msg) Jul 18 15:13:12.273111 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: Jul 18 15:13:12.276608 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f402b20-d5c8-4353-adea-c6036e54012d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:12.313789 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 776/1549] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:12 2022] POST /identity/v3/projects => generated 405 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:12.315416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee4a47-0132-4754-9afb-d64c189547e9 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:12.394169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 774/1550] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:12 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:12.396518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f656d1-bbc1-40f0-a1bd-9f8c88d01248 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.397014 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f656d1-bbc1-40f0-a1bd-9f8c88d01248 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.397499 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f656d1-bbc1-40f0-a1bd-9f8c88d01248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.425333 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-e1f656d1-bbc1-40f0-a1bd-9f8c88d01248 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:12.425735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee4a47-0132-4754-9afb-d64c189547e9 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:12.426718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e1f656d1-bbc1-40f0-a1bd-9f8c88d01248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:12.512398 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d9b93-9aa3-4680-9d17-3c818bbea19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:12.513480 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d9b93-9aa3-4680-9d17-3c818bbea19a admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.513963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d9b93-9aa3-4680-9d17-3c818bbea19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.514456 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d9b93-9aa3-4680-9d17-3c818bbea19a admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.515848 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d9b93-9aa3-4680-9d17-3c818bbea19a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:13:12.516371 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d9b93-9aa3-4680-9d17-3c818bbea19a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:12.517857 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: WARNING py.warnings [None req-5c6d9b93-9aa3-4680-9d17-3c818bbea19a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 15:13:12.517857 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: warnings.warn(msg) Jul 18 15:13:12.517857 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: Jul 18 15:13:12.518880 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 775/1551] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:12.519359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d9b93-9aa3-4680-9d17-3c818bbea19a admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:12.520323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9697d7-9aa6-4088-9fbe-b0044f5e3b4c None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.520646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9697d7-9aa6-4088-9fbe-b0044f5e3b4c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.520953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9697d7-9aa6-4088-9fbe-b0044f5e3b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.543838 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 777/1552] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:13:12.546214 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc96d9-e076-439a-8892-9111c73d42c7 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.546491 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc96d9-e076-439a-8892-9111c73d42c7 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.546824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc96d9-e076-439a-8892-9111c73d42c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.560095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-9f9697d7-9aa6-4088-9fbe-b0044f5e3b4c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:12.562733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9f9697d7-9aa6-4088-9fbe-b0044f5e3b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:12.568675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-a1cc96d9-e076-439a-8892-9111c73d42c7 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:12.572898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a1cc96d9-e076-439a-8892-9111c73d42c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:12.578730 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 776/1553] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:12.580135 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9697d7-9aa6-4088-9fbe-b0044f5e3b4c None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:12.585747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 778/1554] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:12.588834 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d426560-595f-45a1-872f-f54562428bcd None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.589125 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d426560-595f-45a1-872f-f54562428bcd None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.589405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d426560-595f-45a1-872f-f54562428bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.609765 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9697d7-9aa6-4088-9fbe-b0044f5e3b4c None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:12.613961 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-4d426560-595f-45a1-872f-f54562428bcd None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:12.615551 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4d426560-595f-45a1-872f-f54562428bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:12.618524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476410c-a3f2-4efe-b5e7-02554e20528b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:12.619734 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476410c-a3f2-4efe-b5e7-02554e20528b admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.620299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476410c-a3f2-4efe-b5e7-02554e20528b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.620698 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476410c-a3f2-4efe-b5e7-02554e20528b admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.622009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476410c-a3f2-4efe-b5e7-02554e20528b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:12.625839 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476410c-a3f2-4efe-b5e7-02554e20528b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:12.632697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 779/1555] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:12.632697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcbe98-ff16-4865-89fa-149fe55676f3 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.632697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcbe98-ff16-4865-89fa-149fe55676f3 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.632697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcbe98-ff16-4865-89fa-149fe55676f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.669706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-1bdcbe98-ff16-4865-89fa-149fe55676f3 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:12.672014 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1bdcbe98-ff16-4865-89fa-149fe55676f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:12.690357 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 780/1556] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:12.693311 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcbe98-ff16-4865-89fa-149fe55676f3 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:12.715594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcbe98-ff16-4865-89fa-149fe55676f3 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:12.725100 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093be0a-cf27-4570-9d82-6ae289b5451f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:12.726103 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093be0a-cf27-4570-9d82-6ae289b5451f admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.726644 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093be0a-cf27-4570-9d82-6ae289b5451f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.727347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093be0a-cf27-4570-9d82-6ae289b5451f admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.728885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093be0a-cf27-4570-9d82-6ae289b5451f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:12.731392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: WARNING py.warnings [None req-5093be0a-cf27-4570-9d82-6ae289b5451f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 15:13:12.731392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: warnings.warn(msg) Jul 18 15:13:12.731392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: Jul 18 15:13:12.734161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093be0a-cf27-4570-9d82-6ae289b5451f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:12.786041 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 777/1557] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:12 2022] POST /identity/v3/users => generated 303 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:12.788055 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a604f3d-f70c-49be-a6ff-faded4eecbe0 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.788463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a604f3d-f70c-49be-a6ff-faded4eecbe0 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.788936 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a604f3d-f70c-49be-a6ff-faded4eecbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.814028 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-8a604f3d-f70c-49be-a6ff-faded4eecbe0 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:12.816419 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-8a604f3d-f70c-49be-a6ff-faded4eecbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:12.827332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 778/1558] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:12.831873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a604f3d-f70c-49be-a6ff-faded4eecbe0 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:12.840742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a604f3d-f70c-49be-a6ff-faded4eecbe0 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:12.846847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924bb7f-acea-4f21-8080-fa778eabaabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:12.847720 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924bb7f-acea-4f21-8080-fa778eabaabe admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.848094 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924bb7f-acea-4f21-8080-fa778eabaabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.848378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924bb7f-acea-4f21-8080-fa778eabaabe admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.849346 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924bb7f-acea-4f21-8080-fa778eabaabe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:12.851743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924bb7f-acea-4f21-8080-fa778eabaabe admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:12.865022 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 781/1559] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:12 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:12.867031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377913f1-2bff-47bd-9a65-5be6cc7b665c None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.867481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377913f1-2bff-47bd-9a65-5be6cc7b665c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.869190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377913f1-2bff-47bd-9a65-5be6cc7b665c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.893772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-377913f1-2bff-47bd-9a65-5be6cc7b665c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:12.895198 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-377913f1-2bff-47bd-9a65-5be6cc7b665c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:12.907087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 782/1560] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:12.908464 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377913f1-2bff-47bd-9a65-5be6cc7b665c None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:12.923052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377913f1-2bff-47bd-9a65-5be6cc7b665c None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:12.931718 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d790a9-fd04-4ab2-9a57-66063f0f759c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:12.932503 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d790a9-fd04-4ab2-9a57-66063f0f759c admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.942787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d790a9-fd04-4ab2-9a57-66063f0f759c admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.942787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d790a9-fd04-4ab2-9a57-66063f0f759c admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.942787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d790a9-fd04-4ab2-9a57-66063f0f759c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:13:12.942787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d790a9-fd04-4ab2-9a57-66063f0f759c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:12.942787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d790a9-fd04-4ab2-9a57-66063f0f759c admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:12.942787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:12.963272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 783/1561] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:13:12.967683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e82a2-c664-4393-9c3e-5026ea0e4872 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.967683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e82a2-c664-4393-9c3e-5026ea0e4872 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.967683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e82a2-c664-4393-9c3e-5026ea0e4872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:12.985774 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 779/1562] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:12 2022] POST /identity/v3/projects => generated 393 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:12.988376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebbecf-3179-4e3f-a44d-88b37b0766f5 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:12.988703 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebbecf-3179-4e3f-a44d-88b37b0766f5 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:12.988950 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebbecf-3179-4e3f-a44d-88b37b0766f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:13.002938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-252e82a2-c664-4393-9c3e-5026ea0e4872 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:13.007500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-252e82a2-c664-4393-9c3e-5026ea0e4872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:13.022168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-aeebbecf-3179-4e3f-a44d-88b37b0766f5 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:13.024585 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-aeebbecf-3179-4e3f-a44d-88b37b0766f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:13.079089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:13.099800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 784/1563] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:13.101099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e82a2-c664-4393-9c3e-5026ea0e4872 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.115169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e82a2-c664-4393-9c3e-5026ea0e4872 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:13.122987 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d0f0a-5d84-41cf-b16a-7d0d120ea562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:13.123924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d0f0a-5d84-41cf-b16a-7d0d120ea562 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:13.127874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d0f0a-5d84-41cf-b16a-7d0d120ea562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:13.127874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d0f0a-5d84-41cf-b16a-7d0d120ea562 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:13.127874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d0f0a-5d84-41cf-b16a-7d0d120ea562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:13.127874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d0f0a-5d84-41cf-b16a-7d0d120ea562 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:13.131750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 780/1564] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:13.132721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeebbecf-3179-4e3f-a44d-88b37b0766f5 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.146975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 785/1565] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:13.148201 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d0f0a-5d84-41cf-b16a-7d0d120ea562 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.149967 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeebbecf-3179-4e3f-a44d-88b37b0766f5 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:13.159381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd907417-2095-4030-b093-dfc125544e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:13.160242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd907417-2095-4030-b093-dfc125544e27 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:13.160575 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd907417-2095-4030-b093-dfc125544e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:13.160981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd907417-2095-4030-b093-dfc125544e27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:13.161904 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd907417-2095-4030-b093-dfc125544e27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:13:13.162266 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd907417-2095-4030-b093-dfc125544e27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:13.164202 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd907417-2095-4030-b093-dfc125544e27 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:13.164356 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d0f0a-5d84-41cf-b16a-7d0d120ea562 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:13.172103 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51c190-0bfa-4cb7-a664-73762de3ee83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:13.172890 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51c190-0bfa-4cb7-a664-73762de3ee83 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:13.174180 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51c190-0bfa-4cb7-a664-73762de3ee83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:13.174180 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51c190-0bfa-4cb7-a664-73762de3ee83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:13.174956 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51c190-0bfa-4cb7-a664-73762de3ee83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:13.177916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51c190-0bfa-4cb7-a664-73762de3ee83 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:13.191857 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 781/1566] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:13:13.191857 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd907417-2095-4030-b093-dfc125544e27 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.194168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd907417-2095-4030-b093-dfc125544e27 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:13.195403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 786/1567] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:13.197948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51c190-0bfa-4cb7-a664-73762de3ee83 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.204823 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ac1c6-f80f-4171-abd8-584abd9c62f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:13.205946 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ac1c6-f80f-4171-abd8-584abd9c62f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:13.206415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ac1c6-f80f-4171-abd8-584abd9c62f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:13.207081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ac1c6-f80f-4171-abd8-584abd9c62f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:13.208486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ac1c6-f80f-4171-abd8-584abd9c62f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:13:13.209246 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ac1c6-f80f-4171-abd8-584abd9c62f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:13.212498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ac1c6-f80f-4171-abd8-584abd9c62f2 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:13.213851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51c190-0bfa-4cb7-a664-73762de3ee83 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:13.221973 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714359d-5db1-4f4d-897a-04299c6e1b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:13.223088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714359d-5db1-4f4d-897a-04299c6e1b3a admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:13.223519 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714359d-5db1-4f4d-897a-04299c6e1b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:13.223977 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714359d-5db1-4f4d-897a-04299c6e1b3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:13.225238 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714359d-5db1-4f4d-897a-04299c6e1b3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:13.229100 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714359d-5db1-4f4d-897a-04299c6e1b3a admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:13.237506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 782/1568] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:13:13.239164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ac1c6-f80f-4171-abd8-584abd9c62f2 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.268038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ac1c6-f80f-4171-abd8-584abd9c62f2 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:13.275626 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c8ee3-ca79-4e10-9717-a7736eae53c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:13.276433 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c8ee3-ca79-4e10-9717-a7736eae53c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:13.276723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c8ee3-ca79-4e10-9717-a7736eae53c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:13.277092 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c8ee3-ca79-4e10-9717-a7736eae53c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:13.278194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c8ee3-ca79-4e10-9717-a7736eae53c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:13.282086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c8ee3-ca79-4e10-9717-a7736eae53c1 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:13.353768 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 787/1569] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:13 2022] POST /identity/v3/projects => generated 389 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:13.353768 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714359d-5db1-4f4d-897a-04299c6e1b3a admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.423798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 783/1570] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:13 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:13.425958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c8ee3-ca79-4e10-9717-a7736eae53c1 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.436466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714359d-5db1-4f4d-897a-04299c6e1b3a admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:13.442364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87956d67-1d29-475b-a490-11fa1db51654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:13.443139 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87956d67-1d29-475b-a490-11fa1db51654 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:13.443475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87956d67-1d29-475b-a490-11fa1db51654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:13.443860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87956d67-1d29-475b-a490-11fa1db51654 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:13.444357 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c8ee3-ca79-4e10-9717-a7736eae53c1 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:13.444663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87956d67-1d29-475b-a490-11fa1db51654 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:13:13.445142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87956d67-1d29-475b-a490-11fa1db51654 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:13.448342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87956d67-1d29-475b-a490-11fa1db51654 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:13.453828 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469eb6e6-b006-4504-b7c5-dda4a4dfa6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:13.455023 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469eb6e6-b006-4504-b7c5-dda4a4dfa6aa admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:13.455524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469eb6e6-b006-4504-b7c5-dda4a4dfa6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:13.456006 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469eb6e6-b006-4504-b7c5-dda4a4dfa6aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:13.457388 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469eb6e6-b006-4504-b7c5-dda4a4dfa6aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:13:13.457931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469eb6e6-b006-4504-b7c5-dda4a4dfa6aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:13.461305 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469eb6e6-b006-4504-b7c5-dda4a4dfa6aa admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:13.467519 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 788/1571] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:13:13.468788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87956d67-1d29-475b-a490-11fa1db51654 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.490015 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 784/1572] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:13:13.490015 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469eb6e6-b006-4504-b7c5-dda4a4dfa6aa admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.494356 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469eb6e6-b006-4504-b7c5-dda4a4dfa6aa admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:13.497544 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87956d67-1d29-475b-a490-11fa1db51654 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:13.501882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb98e0e-c025-439d-bb29-7b78168790d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:13.503046 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb98e0e-c025-439d-bb29-7b78168790d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:13.503235 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bc4e8-753b-4d9c-a6a2-8e87ab15e331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:13.503545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb98e0e-c025-439d-bb29-7b78168790d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:13.503856 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bc4e8-753b-4d9c-a6a2-8e87ab15e331 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:13.504054 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb98e0e-c025-439d-bb29-7b78168790d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:13.504179 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bc4e8-753b-4d9c-a6a2-8e87ab15e331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:13.504411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bc4e8-753b-4d9c-a6a2-8e87ab15e331 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:13.505187 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bc4e8-753b-4d9c-a6a2-8e87ab15e331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:13.505431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb98e0e-c025-439d-bb29-7b78168790d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:13.507070 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bc4e8-753b-4d9c-a6a2-8e87ab15e331 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:13.510047 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb98e0e-c025-439d-bb29-7b78168790d8 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:13.521596 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 789/1573] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:13.522714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bc4e8-753b-4d9c-a6a2-8e87ab15e331 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.534273 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bc4e8-753b-4d9c-a6a2-8e87ab15e331 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:13.539671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d7792-eab3-42d3-88ad-de89ba9e3699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:13.540554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d7792-eab3-42d3-88ad-de89ba9e3699 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:13.541101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d7792-eab3-42d3-88ad-de89ba9e3699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:13.541391 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d7792-eab3-42d3-88ad-de89ba9e3699 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:13.542167 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d7792-eab3-42d3-88ad-de89ba9e3699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:13.544217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d7792-eab3-42d3-88ad-de89ba9e3699 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:13.559133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 790/1574] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:13.560267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d7792-eab3-42d3-88ad-de89ba9e3699 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.573304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d7792-eab3-42d3-88ad-de89ba9e3699 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:13.578951 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdec60-5ad4-4b3b-b307-ce47d5b44aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:13.579654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdec60-5ad4-4b3b-b307-ce47d5b44aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:13.579941 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdec60-5ad4-4b3b-b307-ce47d5b44aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:13.580219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdec60-5ad4-4b3b-b307-ce47d5b44aa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:13.580938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdec60-5ad4-4b3b-b307-ce47d5b44aa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:13.583063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdec60-5ad4-4b3b-b307-ce47d5b44aa1 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:13.620205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 785/1575] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:13 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:13.621336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb98e0e-c025-439d-bb29-7b78168790d8 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.699803 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:13.708597 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 791/1576] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:13 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:13.710447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdec60-5ad4-4b3b-b307-ce47d5b44aa1 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.713191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb98e0e-c025-439d-bb29-7b78168790d8 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:13.809805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d540d-1a32-4c35-aae3-a8b14dd8d943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:13.810975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d540d-1a32-4c35-aae3-a8b14dd8d943 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:13.811176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdec60-5ad4-4b3b-b307-ce47d5b44aa1 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:13.811843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d540d-1a32-4c35-aae3-a8b14dd8d943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:13.812034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d540d-1a32-4c35-aae3-a8b14dd8d943 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:13.812892 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d540d-1a32-4c35-aae3-a8b14dd8d943 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:13.816079 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d540d-1a32-4c35-aae3-a8b14dd8d943 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:13.816932 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583a34c-463c-4f69-a219-354c98bb26ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:13.817771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583a34c-463c-4f69-a219-354c98bb26ae admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:13.819533 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583a34c-463c-4f69-a219-354c98bb26ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:13.819804 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583a34c-463c-4f69-a219-354c98bb26ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:13.820566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583a34c-463c-4f69-a219-354c98bb26ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:13.823657 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583a34c-463c-4f69-a219-354c98bb26ae admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:13.829090 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:13.836370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 792/1577] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:13.838252 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583a34c-463c-4f69-a219-354c98bb26ae admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.850684 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583a34c-463c-4f69-a219-354c98bb26ae admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:13.856595 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d965be-0c95-4426-a883-25a7548df77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:13.857285 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d965be-0c95-4426-a883-25a7548df77f admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:13.857563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d965be-0c95-4426-a883-25a7548df77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:13.857839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d965be-0c95-4426-a883-25a7548df77f admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:13.858595 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d965be-0c95-4426-a883-25a7548df77f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:13.860769 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d965be-0c95-4426-a883-25a7548df77f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:13.969144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 786/1578] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:13 2022] POST /identity/v3/users => generated 295 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:13.969144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d540d-1a32-4c35-aae3-a8b14dd8d943 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.971354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 793/1579] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:13 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:13.972612 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d965be-0c95-4426-a883-25a7548df77f admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:13.986204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d540d-1a32-4c35-aae3-a8b14dd8d943 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:14.081155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959cbc4-0fab-4f38-b285-fb6ff1f38a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:14.082634 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959cbc4-0fab-4f38-b285-fb6ff1f38a8b admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:14.083016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959cbc4-0fab-4f38-b285-fb6ff1f38a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:14.083490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959cbc4-0fab-4f38-b285-fb6ff1f38a8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:14.083667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d965be-0c95-4426-a883-25a7548df77f admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:14.085005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959cbc4-0fab-4f38-b285-fb6ff1f38a8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:14.089037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959cbc4-0fab-4f38-b285-fb6ff1f38a8b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:14.093644 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a7bab-cf86-499f-8328-5cabbd4b7896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:14.095172 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a7bab-cf86-499f-8328-5cabbd4b7896 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:14.095638 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a7bab-cf86-499f-8328-5cabbd4b7896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:14.096091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a7bab-cf86-499f-8328-5cabbd4b7896 admin admin] PATH_INFO: `/v3/projects/c9e531c14bb746679c26077041def6ab/users/a4dfa2bdf4ba49938251c66cc8c9b604/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:14.121102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a7bab-cf86-499f-8328-5cabbd4b7896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e531c14bb746679c26077041def6ab, user_id=a4dfa2bdf4ba49938251c66cc8c9b604, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:14.123985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a7bab-cf86-499f-8328-5cabbd4b7896 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:14.231295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 787/1580] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:13 2022] POST /identity/v3/projects => generated 383 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:14.231976 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959cbc4-0fab-4f38-b285-fb6ff1f38a8b admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:14.248992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 794/1581] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:13 2022] PUT /identity/v3/projects/c9e531c14bb746679c26077041def6ab/users/a4dfa2bdf4ba49938251c66cc8c9b604/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:14.249797 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a7bab-cf86-499f-8328-5cabbd4b7896 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:14.322808 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959cbc4-0fab-4f38-b285-fb6ff1f38a8b admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:14.328572 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc4ef5-0a0b-4e6c-8a7f-6ba6670709c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:14.329638 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:14.329808 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc4ef5-0a0b-4e6c-8a7f-6ba6670709c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:14.330155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc4ef5-0a0b-4e6c-8a7f-6ba6670709c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:14.330440 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc4ef5-0a0b-4e6c-8a7f-6ba6670709c7 admin admin] PATH_INFO: `/v3/projects/7cb30e124ce7458f9c29e6c6ae3d95f0/users/22b86eef7820462fa4e6cbb22f30b853/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:14.335283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a7bab-cf86-499f-8328-5cabbd4b7896 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:14.341694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44656ac-e73b-4247-b9f1-244d8a14f717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:14.342438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44656ac-e73b-4247-b9f1-244d8a14f717 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:14.342784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44656ac-e73b-4247-b9f1-244d8a14f717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:14.343099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44656ac-e73b-4247-b9f1-244d8a14f717 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:14.343876 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44656ac-e73b-4247-b9f1-244d8a14f717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:14.346438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44656ac-e73b-4247-b9f1-244d8a14f717 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:14.351714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc4ef5-0a0b-4e6c-8a7f-6ba6670709c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cb30e124ce7458f9c29e6c6ae3d95f0, user_id=22b86eef7820462fa4e6cbb22f30b853, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:14.354124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: WARNING py.warnings [None req-d0dc4ef5-0a0b-4e6c-8a7f-6ba6670709c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 15:13:14.354124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: warnings.warn(msg) Jul 18 15:13:14.354124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: Jul 18 15:13:14.357823 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc4ef5-0a0b-4e6c-8a7f-6ba6670709c7 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:14.537281 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 795/1582] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:14 2022] POST /identity/v3/users => generated 290 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:14.538118 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44656ac-e73b-4247-b9f1-244d8a14f717 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:14.551379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44656ac-e73b-4247-b9f1-244d8a14f717 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:14.556995 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9359563-f3b4-4c65-b31a-59f57f302459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:14.557687 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9359563-f3b4-4c65-b31a-59f57f302459 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:14.557993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9359563-f3b4-4c65-b31a-59f57f302459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:14.558245 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9359563-f3b4-4c65-b31a-59f57f302459 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:14.559000 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9359563-f3b4-4c65-b31a-59f57f302459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:14.560985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9359563-f3b4-4c65-b31a-59f57f302459 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:14.594802 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 788/1583] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:14 2022] PUT /identity/v3/projects/7cb30e124ce7458f9c29e6c6ae3d95f0/users/22b86eef7820462fa4e6cbb22f30b853/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:14.595850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc4ef5-0a0b-4e6c-8a7f-6ba6670709c7 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:14.670073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc4ef5-0a0b-4e6c-8a7f-6ba6670709c7 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:14.675027 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e91994-d56c-4ce2-9dd1-bc0371be1e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:14.675757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e91994-d56c-4ce2-9dd1-bc0371be1e39 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:14.676034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e91994-d56c-4ce2-9dd1-bc0371be1e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:14.676309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e91994-d56c-4ce2-9dd1-bc0371be1e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:14.677057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e91994-d56c-4ce2-9dd1-bc0371be1e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:14.678795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e91994-d56c-4ce2-9dd1-bc0371be1e39 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:14.680734 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:14.691866 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 789/1584] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:14.692773 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e91994-d56c-4ce2-9dd1-bc0371be1e39 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:14.706010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e91994-d56c-4ce2-9dd1-bc0371be1e39 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:14.712449 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9b76a-c3c5-43a8-a060-4590168e3445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:14.713151 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9b76a-c3c5-43a8-a060-4590168e3445 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:14.713440 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9b76a-c3c5-43a8-a060-4590168e3445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:14.713724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9b76a-c3c5-43a8-a060-4590168e3445 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:14.714495 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9b76a-c3c5-43a8-a060-4590168e3445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:14.716178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9b76a-c3c5-43a8-a060-4590168e3445 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:14.731541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 790/1585] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:14.731541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9b76a-c3c5-43a8-a060-4590168e3445 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:14.739786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9b76a-c3c5-43a8-a060-4590168e3445 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:14.744317 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 796/1586] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:14 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:14.747836 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9359563-f3b4-4c65-b31a-59f57f302459 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:14.747959 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29239e2-bf2a-4685-87f1-4d9785656726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:14.747959 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29239e2-bf2a-4685-87f1-4d9785656726 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:14.747959 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29239e2-bf2a-4685-87f1-4d9785656726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:14.748250 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29239e2-bf2a-4685-87f1-4d9785656726 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:14.749487 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29239e2-bf2a-4685-87f1-4d9785656726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:14.752681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29239e2-bf2a-4685-87f1-4d9785656726 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:14.757674 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9359563-f3b4-4c65-b31a-59f57f302459 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:14.764609 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0c1d9-76ae-46aa-b9b2-13af42f5ed5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:14.765340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0c1d9-76ae-46aa-b9b2-13af42f5ed5b admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:14.765687 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0c1d9-76ae-46aa-b9b2-13af42f5ed5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:14.765991 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0c1d9-76ae-46aa-b9b2-13af42f5ed5b admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:14.766771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0c1d9-76ae-46aa-b9b2-13af42f5ed5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:14.768830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0c1d9-76ae-46aa-b9b2-13af42f5ed5b admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:14.853780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 791/1587] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:14 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:14.854985 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6de72-b699-4fbe-a7af-c662cf536244 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:14.855494 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6de72-b699-4fbe-a7af-c662cf536244 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:14.855494 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6de72-b699-4fbe-a7af-c662cf536244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:14.876815 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-31e6de72-b699-4fbe-a7af-c662cf536244 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:14.878117 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-31e6de72-b699-4fbe-a7af-c662cf536244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:14.887577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 797/1588] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:14 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:14.896828 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 792/1589] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:14.897859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0c1d9-76ae-46aa-b9b2-13af42f5ed5b admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:14.898038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f395af-bc0b-4225-8f07-315a0ea55ac7 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:14.898038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f395af-bc0b-4225-8f07-315a0ea55ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:14.898038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f395af-bc0b-4225-8f07-315a0ea55ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:14.905350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0c1d9-76ae-46aa-b9b2-13af42f5ed5b admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:14.911350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6623849-8783-4d5b-8f2a-c6b6e14191ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:14.912115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6623849-8783-4d5b-8f2a-c6b6e14191ef admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:14.912467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6623849-8783-4d5b-8f2a-c6b6e14191ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:14.912748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6623849-8783-4d5b-8f2a-c6b6e14191ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:14.913592 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6623849-8783-4d5b-8f2a-c6b6e14191ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:14.916295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6623849-8783-4d5b-8f2a-c6b6e14191ef admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:14.919494 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-a7f395af-bc0b-4225-8f07-315a0ea55ac7 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:14.920786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a7f395af-bc0b-4225-8f07-315a0ea55ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:14.929358 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 793/1590] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:14.930258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f395af-bc0b-4225-8f07-315a0ea55ac7 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:14.932977 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 798/1591] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:14.933934 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6623849-8783-4d5b-8f2a-c6b6e14191ef admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:14.951478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f395af-bc0b-4225-8f07-315a0ea55ac7 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:14.957408 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280b74e-62a9-4b7e-afbd-4571114fcaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:14.958707 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280b74e-62a9-4b7e-afbd-4571114fcaf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:14.959374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280b74e-62a9-4b7e-afbd-4571114fcaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:14.959834 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280b74e-62a9-4b7e-afbd-4571114fcaf7 admin admin] PATH_INFO: `/v3/projects/0e83fff7c7524da9905cdb86ccdb565a/users/648c869550ee437a8e775e997c741e58/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:14.959991 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6623849-8783-4d5b-8f2a-c6b6e14191ef admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:14.968958 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61d8e1-62a6-470a-a6b9-31e472ec584d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:14.970148 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61d8e1-62a6-470a-a6b9-31e472ec584d admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:14.970654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61d8e1-62a6-470a-a6b9-31e472ec584d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:14.971147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61d8e1-62a6-470a-a6b9-31e472ec584d admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:14.972484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61d8e1-62a6-470a-a6b9-31e472ec584d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:14.975510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61d8e1-62a6-470a-a6b9-31e472ec584d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:14.980158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280b74e-62a9-4b7e-afbd-4571114fcaf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e83fff7c7524da9905cdb86ccdb565a, user_id=648c869550ee437a8e775e997c741e58, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:14.984356 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280b74e-62a9-4b7e-afbd-4571114fcaf7 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:14.997615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 799/1592] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:15.000006 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61d8e1-62a6-470a-a6b9-31e472ec584d admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:15.014191 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61d8e1-62a6-470a-a6b9-31e472ec584d admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:15.019579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835bfd2d-2783-4fe9-8f30-f36d6d3392ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:15.020315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bfd2d-2783-4fe9-8f30-f36d6d3392ba admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:15.020649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bfd2d-2783-4fe9-8f30-f36d6d3392ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:15.020909 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bfd2d-2783-4fe9-8f30-f36d6d3392ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:15.021663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835bfd2d-2783-4fe9-8f30-f36d6d3392ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:15.023371 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835bfd2d-2783-4fe9-8f30-f36d6d3392ba admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:15.035931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 800/1593] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:15.037219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835bfd2d-2783-4fe9-8f30-f36d6d3392ba admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:15.049003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835bfd2d-2783-4fe9-8f30-f36d6d3392ba admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:15.056274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef50a7-bc4a-4bb6-877d-daa04df153c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:15.057187 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef50a7-bc4a-4bb6-877d-daa04df153c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:15.057715 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef50a7-bc4a-4bb6-877d-daa04df153c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:15.058134 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef50a7-bc4a-4bb6-877d-daa04df153c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:15.059331 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef50a7-bc4a-4bb6-877d-daa04df153c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:15.063361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 794/1594] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:14 2022] PUT /identity/v3/projects/0e83fff7c7524da9905cdb86ccdb565a/users/648c869550ee437a8e775e997c741e58/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:15.063592 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef50a7-bc4a-4bb6-877d-daa04df153c3 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:15.063745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280b74e-62a9-4b7e-afbd-4571114fcaf7 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:15.086031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 801/1595] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:15.086031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef50a7-bc4a-4bb6-877d-daa04df153c3 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:15.087938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-41ef50a7-bc4a-4bb6-877d-daa04df153c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:15.180610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280b74e-62a9-4b7e-afbd-4571114fcaf7 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:15.182803 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef50a7-bc4a-4bb6-877d-daa04df153c3 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:15.186662 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec709f-d148-4738-baae-a565cc1ffe6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:15.187535 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec709f-d148-4738-baae-a565cc1ffe6d admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:15.188171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec709f-d148-4738-baae-a565cc1ffe6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:15.188451 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec709f-d148-4738-baae-a565cc1ffe6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:15.189366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec709f-d148-4738-baae-a565cc1ffe6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:15.191457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec709f-d148-4738-baae-a565cc1ffe6d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:15.204984 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 795/1596] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:15.205981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec709f-d148-4738-baae-a565cc1ffe6d admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:15.218828 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec709f-d148-4738-baae-a565cc1ffe6d admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:15.224476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b602215-abd3-42cb-aa10-85627aaea650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:15.224826 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b602215-abd3-42cb-aa10-85627aaea650 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:15.225114 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b602215-abd3-42cb-aa10-85627aaea650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:15.225397 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b602215-abd3-42cb-aa10-85627aaea650 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:15.226142 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b602215-abd3-42cb-aa10-85627aaea650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:15.228375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b602215-abd3-42cb-aa10-85627aaea650 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:15.245879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 796/1597] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:15.246866 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b602215-abd3-42cb-aa10-85627aaea650 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:15.267904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bb688-20da-4726-bc23-940d052ad8fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:15.269156 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bb688-20da-4726-bc23-940d052ad8fe service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:15.269687 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bb688-20da-4726-bc23-940d052ad8fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:15.270197 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bb688-20da-4726-bc23-940d052ad8fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:15.285543 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bb688-20da-4726-bc23-940d052ad8fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:15.287537 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bb688-20da-4726-bc23-940d052ad8fe service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:15.304032 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 802/1598] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:15.306407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bb688-20da-4726-bc23-940d052ad8fe service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:15.319615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bb688-20da-4726-bc23-940d052ad8fe service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:15.327570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bba6c4-786d-45ed-b98e-4ef32c843e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:15.328741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bba6c4-786d-45ed-b98e-4ef32c843e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:15.329308 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bba6c4-786d-45ed-b98e-4ef32c843e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:15.330184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bba6c4-786d-45ed-b98e-4ef32c843e16 admin admin] PATH_INFO: `/v3/projects/ea95f701696f4c28b2d4bbef9d0af45d/users/f3f0c37e426640f9960c1da232605e55/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:15.353738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b602215-abd3-42cb-aa10-85627aaea650 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:15.358330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4d511-41d2-48b2-be9b-5bf3e4f217af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:15.359031 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4d511-41d2-48b2-be9b-5bf3e4f217af service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:15.359307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4d511-41d2-48b2-be9b-5bf3e4f217af service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:15.359615 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4d511-41d2-48b2-be9b-5bf3e4f217af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:15.360411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bba6c4-786d-45ed-b98e-4ef32c843e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea95f701696f4c28b2d4bbef9d0af45d, user_id=f3f0c37e426640f9960c1da232605e55, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:15.364576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bba6c4-786d-45ed-b98e-4ef32c843e16 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:15.374245 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4d511-41d2-48b2-be9b-5bf3e4f217af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:15.380634 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4d511-41d2-48b2-be9b-5bf3e4f217af service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:15.390452 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 797/1599] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:15.392143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4d511-41d2-48b2-be9b-5bf3e4f217af service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:15.410387 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4d511-41d2-48b2-be9b-5bf3e4f217af service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:15.421506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1715ba2-de94-453f-ad29-1435531ae106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:15.437618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1715ba2-de94-453f-ad29-1435531ae106 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:15.437618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1715ba2-de94-453f-ad29-1435531ae106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:15.437618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1715ba2-de94-453f-ad29-1435531ae106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:15.437618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1715ba2-de94-453f-ad29-1435531ae106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:15.437618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1715ba2-de94-453f-ad29-1435531ae106 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:15.467097 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 798/1600] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:15.467097 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082032dd-1d92-4c7f-91e3-091260a2fd20 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:15.467097 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082032dd-1d92-4c7f-91e3-091260a2fd20 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:15.467097 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082032dd-1d92-4c7f-91e3-091260a2fd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:15.492345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:15.500322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-082032dd-1d92-4c7f-91e3-091260a2fd20 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:15.502655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-082032dd-1d92-4c7f-91e3-091260a2fd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:15.524474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 799/1601] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:15.525724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082032dd-1d92-4c7f-91e3-091260a2fd20 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:15.547565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082032dd-1d92-4c7f-91e3-091260a2fd20 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:15.562410 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900e142-79d8-4e1d-a9f3-f432fac573f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:15.563200 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900e142-79d8-4e1d-a9f3-f432fac573f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:15.569323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900e142-79d8-4e1d-a9f3-f432fac573f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:15.569323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900e142-79d8-4e1d-a9f3-f432fac573f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:15.569579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 803/1602] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:15 2022] PUT /identity/v3/projects/ea95f701696f4c28b2d4bbef9d0af45d/users/f3f0c37e426640f9960c1da232605e55/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:15.569579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bba6c4-786d-45ed-b98e-4ef32c843e16 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:15.569890 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900e142-79d8-4e1d-a9f3-f432fac573f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:15.569890 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900e142-79d8-4e1d-a9f3-f432fac573f1 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:15.575824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:15.597754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 800/1603] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:15.599955 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900e142-79d8-4e1d-a9f3-f432fac573f1 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:15.620000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900e142-79d8-4e1d-a9f3-f432fac573f1 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:15.674747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bba6c4-786d-45ed-b98e-4ef32c843e16 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:15.681753 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d3022-0542-44a9-8de2-37c23e01b42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:15.682513 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d3022-0542-44a9-8de2-37c23e01b42e admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:15.682859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d3022-0542-44a9-8de2-37c23e01b42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:15.683974 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d3022-0542-44a9-8de2-37c23e01b42e admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:15.684121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d3022-0542-44a9-8de2-37c23e01b42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:15.685982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d3022-0542-44a9-8de2-37c23e01b42e admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:15.700108 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 804/1604] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:15.701304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d3022-0542-44a9-8de2-37c23e01b42e admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:15.715288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d3022-0542-44a9-8de2-37c23e01b42e admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:15.726623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac9edc-317e-495e-9e21-3c7423e33609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:15.729419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac9edc-317e-495e-9e21-3c7423e33609 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:15.729871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac9edc-317e-495e-9e21-3c7423e33609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:15.730270 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac9edc-317e-495e-9e21-3c7423e33609 admin admin] PATH_INFO: `/v3/projects/c5023c967d9841d29cc721efe9684b89/users/eca68707420847969407a77dbb6bc67c/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:15.749041 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac9edc-317e-495e-9e21-3c7423e33609 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5023c967d9841d29cc721efe9684b89, user_id=eca68707420847969407a77dbb6bc67c, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:15.752276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac9edc-317e-495e-9e21-3c7423e33609 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:15.753860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce726af-c0c9-4ff0-a1c8-92c0f98df364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:15.755745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce726af-c0c9-4ff0-a1c8-92c0f98df364 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:15.756593 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce726af-c0c9-4ff0-a1c8-92c0f98df364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:15.757379 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce726af-c0c9-4ff0-a1c8-92c0f98df364 admin admin] PATH_INFO: `/v3/projects/7d9e87e7a63642338bfa2ccc3110f27c/users/36d2ee1e945d481a946336506846f88b/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:15.802461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce726af-c0c9-4ff0-a1c8-92c0f98df364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d9e87e7a63642338bfa2ccc3110f27c, user_id=36d2ee1e945d481a946336506846f88b, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:15.808049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce726af-c0c9-4ff0-a1c8-92c0f98df364 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:15.838353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 805/1605] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:15 2022] PUT /identity/v3/projects/c5023c967d9841d29cc721efe9684b89/users/eca68707420847969407a77dbb6bc67c/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:15.840109 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac9edc-317e-495e-9e21-3c7423e33609 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:15.916855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 801/1606] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:15 2022] PUT /identity/v3/projects/7d9e87e7a63642338bfa2ccc3110f27c/users/36d2ee1e945d481a946336506846f88b/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:15.919500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57123e-1c78-49c7-9e89-c4a3e7f72186 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:15.921028 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57123e-1c78-49c7-9e89-c4a3e7f72186 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:15.921783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57123e-1c78-49c7-9e89-c4a3e7f72186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:15.959899 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-9f57123e-1c78-49c7-9e89-c4a3e7f72186 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:15.960380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac9edc-317e-495e-9e21-3c7423e33609 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:15.964528 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9f57123e-1c78-49c7-9e89-c4a3e7f72186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:16.069695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51181b4e-204a-46f4-b280-76cb9a388ff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:16.070044 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 802/1607] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:16.076688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57123e-1c78-49c7-9e89-c4a3e7f72186 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:16.076996 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51181b4e-204a-46f4-b280-76cb9a388ff8 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:16.076996 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51181b4e-204a-46f4-b280-76cb9a388ff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:16.076996 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51181b4e-204a-46f4-b280-76cb9a388ff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:16.091044 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57123e-1c78-49c7-9e89-c4a3e7f72186 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:16.097749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0afab-c3eb-4607-b4c1-6ae9bc9fbbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:16.098715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0afab-c3eb-4607-b4c1-6ae9bc9fbbfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:16.101558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0afab-c3eb-4607-b4c1-6ae9bc9fbbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:16.101558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0afab-c3eb-4607-b4c1-6ae9bc9fbbfa admin admin] PATH_INFO: `/v3/projects/9a337533276641a2b556a55fdab4a049/users/21de90974572419e906e7ac5d9d8292d/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:16.101992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51181b4e-204a-46f4-b280-76cb9a388ff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:16.107750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51181b4e-204a-46f4-b280-76cb9a388ff8 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:16.126832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0afab-c3eb-4607-b4c1-6ae9bc9fbbfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a337533276641a2b556a55fdab4a049, user_id=21de90974572419e906e7ac5d9d8292d, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:16.134506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0afab-c3eb-4607-b4c1-6ae9bc9fbbfa admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:16.138884 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 806/1608] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:16.142470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51181b4e-204a-46f4-b280-76cb9a388ff8 service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:16.167570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51181b4e-204a-46f4-b280-76cb9a388ff8 service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:16.175479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45c323-7832-416c-b9c2-1f40a6332156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:16.176852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45c323-7832-416c-b9c2-1f40a6332156 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:16.177388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45c323-7832-416c-b9c2-1f40a6332156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:16.177778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45c323-7832-416c-b9c2-1f40a6332156 admin admin] PATH_INFO: `/v3/projects/8058c29fd4e04cf9a4e29dc603ce9a38/users/a807005d762c4989900ef5fadad37670/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:16.202000 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45c323-7832-416c-b9c2-1f40a6332156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8058c29fd4e04cf9a4e29dc603ce9a38, user_id=a807005d762c4989900ef5fadad37670, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:16.215953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45c323-7832-416c-b9c2-1f40a6332156 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:16.294223 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 803/1609] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:16 2022] PUT /identity/v3/projects/9a337533276641a2b556a55fdab4a049/users/21de90974572419e906e7ac5d9d8292d/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:16.296094 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c9d2d-1aa5-4df9-8439-1657e0584c2f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:16.296094 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c9d2d-1aa5-4df9-8439-1657e0584c2f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:16.296441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c9d2d-1aa5-4df9-8439-1657e0584c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:16.347097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 807/1610] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:16 2022] PUT /identity/v3/projects/8058c29fd4e04cf9a4e29dc603ce9a38/users/a807005d762c4989900ef5fadad37670/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:16.349016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9dbd8-f008-421e-8fac-a0622cfe24c6 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:16.349646 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9dbd8-f008-421e-8fac-a0622cfe24c6 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:16.350084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9dbd8-f008-421e-8fac-a0622cfe24c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:16.363907 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-0a2c9d2d-1aa5-4df9-8439-1657e0584c2f None None] MFA Rules not processed for user `eca68707420847969407a77dbb6bc67c`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:16.369146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0a2c9d2d-1aa5-4df9-8439-1657e0584c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:16.378181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:16.398066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-11f9dbd8-f008-421e-8fac-a0622cfe24c6 None None] MFA Rules not processed for user `36d2ee1e945d481a946336506846f88b`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:16.401156 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-11f9dbd8-f008-421e-8fac-a0622cfe24c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:16.480363 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:16.553854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 804/1611] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:16.556609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c9d2d-1aa5-4df9-8439-1657e0584c2f None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:16.575928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 808/1612] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:16.587129 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06346b4b-c7ef-4302-9354-2348290d62c8 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:16.587129 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06346b4b-c7ef-4302-9354-2348290d62c8 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:16.587129 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06346b4b-c7ef-4302-9354-2348290d62c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:16.644340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-06346b4b-c7ef-4302-9354-2348290d62c8 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:16.647038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-06346b4b-c7ef-4302-9354-2348290d62c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:16.735943 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c9d2d-1aa5-4df9-8439-1657e0584c2f None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:16.762480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46f53c-09e6-4f62-99da-54d2b00c55dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:16.765704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46f53c-09e6-4f62-99da-54d2b00c55dd service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:16.767038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46f53c-09e6-4f62-99da-54d2b00c55dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:16.767966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46f53c-09e6-4f62-99da-54d2b00c55dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:16.823153 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 809/1613] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:16.831413 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35cbb8-ac30-4cdb-8c49-f40c0440006b None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:16.832574 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35cbb8-ac30-4cdb-8c49-f40c0440006b None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:16.833484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35cbb8-ac30-4cdb-8c49-f40c0440006b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:16.905454 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-2a35cbb8-ac30-4cdb-8c49-f40c0440006b None None] MFA Rules not processed for user `a807005d762c4989900ef5fadad37670`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:16.907424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-2a35cbb8-ac30-4cdb-8c49-f40c0440006b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:17.056702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46f53c-09e6-4f62-99da-54d2b00c55dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:17.061807 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46f53c-09e6-4f62-99da-54d2b00c55dd service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:17.080694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 810/1614] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:17.082414 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35cbb8-ac30-4cdb-8c49-f40c0440006b None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:17.087817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 805/1615] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:17.090484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46f53c-09e6-4f62-99da-54d2b00c55dd service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:17.110943 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46f53c-09e6-4f62-99da-54d2b00c55dd service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:17.121668 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fc324-2052-4c02-b4e5-55250a59ec84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:17.123694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fc324-2052-4c02-b4e5-55250a59ec84 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:17.124593 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fc324-2052-4c02-b4e5-55250a59ec84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:17.125377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fc324-2052-4c02-b4e5-55250a59ec84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:17.127787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fc324-2052-4c02-b4e5-55250a59ec84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:17.132360 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fc324-2052-4c02-b4e5-55250a59ec84 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:17.160778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:17.226525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35cbb8-ac30-4cdb-8c49-f40c0440006b None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:17.233955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269e426-e6e8-4291-b3f7-ec1a75c527ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:17.235119 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269e426-e6e8-4291-b3f7-ec1a75c527ae service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:17.235607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269e426-e6e8-4291-b3f7-ec1a75c527ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:17.236092 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269e426-e6e8-4291-b3f7-ec1a75c527ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:17.257723 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269e426-e6e8-4291-b3f7-ec1a75c527ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:17.261232 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269e426-e6e8-4291-b3f7-ec1a75c527ae service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:17.273219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 806/1616] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:17 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:17.275827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fc324-2052-4c02-b4e5-55250a59ec84 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:17.285020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 811/1617] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:17 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:17.287143 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269e426-e6e8-4291-b3f7-ec1a75c527ae service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:17.330149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269e426-e6e8-4291-b3f7-ec1a75c527ae service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:17.409997 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fc324-2052-4c02-b4e5-55250a59ec84 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:17.419198 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddac206-bd1b-4382-bceb-0880ea43c319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:17.422293 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddac206-bd1b-4382-bceb-0880ea43c319 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:17.422293 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddac206-bd1b-4382-bceb-0880ea43c319 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:17.423523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddac206-bd1b-4382-bceb-0880ea43c319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:17.480294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:17.520466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b02a31-4d18-4630-835c-d46687e1c8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:17.521675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b02a31-4d18-4630-835c-d46687e1c8f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:17.522403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b02a31-4d18-4630-835c-d46687e1c8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:17.522972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b02a31-4d18-4630-835c-d46687e1c8f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:17.528159 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b02a31-4d18-4630-835c-d46687e1c8f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:17.535685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b02a31-4d18-4630-835c-d46687e1c8f8 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:17.607838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddac206-bd1b-4382-bceb-0880ea43c319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:17.609559 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddac206-bd1b-4382-bceb-0880ea43c319 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:17.646573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 807/1618] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:17.648815 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddac206-bd1b-4382-bceb-0880ea43c319 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:17.684219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddac206-bd1b-4382-bceb-0880ea43c319 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:17.699311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831608c-662e-4e1c-b3b0-88d691705983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:17.701933 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831608c-662e-4e1c-b3b0-88d691705983 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:17.701933 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831608c-662e-4e1c-b3b0-88d691705983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:17.701933 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831608c-662e-4e1c-b3b0-88d691705983 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:17.705242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831608c-662e-4e1c-b3b0-88d691705983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:17.707767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 812/1619] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:17 2022] POST /identity/v3/projects => generated 387 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:17.713602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831608c-662e-4e1c-b3b0-88d691705983 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:17.721078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b02a31-4d18-4630-835c-d46687e1c8f8 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:17.856309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 808/1620] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:17 2022] POST /identity/v3/users => generated 297 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:17.862212 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831608c-662e-4e1c-b3b0-88d691705983 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:17.878076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b02a31-4d18-4630-835c-d46687e1c8f8 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:17.881623 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831608c-662e-4e1c-b3b0-88d691705983 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:17.891545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ea2af-bc53-4547-bacb-6ab8841c04d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:17.892770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ea2af-bc53-4547-bacb-6ab8841c04d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:17.893151 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ea2af-bc53-4547-bacb-6ab8841c04d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:17.893543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ea2af-bc53-4547-bacb-6ab8841c04d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:17.894577 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ea2af-bc53-4547-bacb-6ab8841c04d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:17.898099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ea2af-bc53-4547-bacb-6ab8841c04d5 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:17.907492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3dd0f-91eb-48e4-87b8-7354d11c83a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:17.914927 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3dd0f-91eb-48e4-87b8-7354d11c83a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:17.916301 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3dd0f-91eb-48e4-87b8-7354d11c83a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:17.917050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3dd0f-91eb-48e4-87b8-7354d11c83a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:17.918894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3dd0f-91eb-48e4-87b8-7354d11c83a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:17.926801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 809/1621] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:17.933089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ea2af-bc53-4547-bacb-6ab8841c04d5 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:17.954614 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3dd0f-91eb-48e4-87b8-7354d11c83a9 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:17.981111 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ea2af-bc53-4547-bacb-6ab8841c04d5 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:17.996294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bbc79-718a-41b8-aaef-a5848b07d6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:17.998624 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bbc79-718a-41b8-aaef-a5848b07d6a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:17.999915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bbc79-718a-41b8-aaef-a5848b07d6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:18.004316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bbc79-718a-41b8-aaef-a5848b07d6a3 admin admin] PATH_INFO: `/v3/projects/f0e3e1902b02445c8a328a881d9beecd/users/8d7958c8e947445280f051508f756e5d/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:18.029179 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bbc79-718a-41b8-aaef-a5848b07d6a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e3e1902b02445c8a328a881d9beecd, user_id=8d7958c8e947445280f051508f756e5d, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:18.037633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bbc79-718a-41b8-aaef-a5848b07d6a3 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:18.176558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 810/1622] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:17 2022] PUT /identity/v3/projects/f0e3e1902b02445c8a328a881d9beecd/users/8d7958c8e947445280f051508f756e5d/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:18.177642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 813/1623] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:17 2022] POST /identity/v3/users => generated 300 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:18.179654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3dd0f-91eb-48e4-87b8-7354d11c83a9 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:18.183234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bbc79-718a-41b8-aaef-a5848b07d6a3 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:18.199802 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3dd0f-91eb-48e4-87b8-7354d11c83a9 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:18.304583 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:18.366778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5872d0-7c06-4207-9d8c-6d593ca18ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:18.369214 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5872d0-7c06-4207-9d8c-6d593ca18ace admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:18.369826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5872d0-7c06-4207-9d8c-6d593ca18ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:18.370520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5872d0-7c06-4207-9d8c-6d593ca18ace admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:18.377552 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5872d0-7c06-4207-9d8c-6d593ca18ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:18.386743 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5872d0-7c06-4207-9d8c-6d593ca18ace admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:18.440181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 814/1624] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:18.451253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5872d0-7c06-4207-9d8c-6d593ca18ace admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:18.451460 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bbc79-718a-41b8-aaef-a5848b07d6a3 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:18.470282 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94530b-370d-493d-9403-0ac17b6781ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:18.472290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94530b-370d-493d-9403-0ac17b6781ec admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:18.473442 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5872d0-7c06-4207-9d8c-6d593ca18ace admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:18.473987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94530b-370d-493d-9403-0ac17b6781ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:18.474513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94530b-370d-493d-9403-0ac17b6781ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:18.478370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94530b-370d-493d-9403-0ac17b6781ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:18.484882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94530b-370d-493d-9403-0ac17b6781ec admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:18.491767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4887f0-2493-46b0-9574-9fd6fe2d6414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:18.494790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4887f0-2493-46b0-9574-9fd6fe2d6414 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:18.501163 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4887f0-2493-46b0-9574-9fd6fe2d6414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:18.501163 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4887f0-2493-46b0-9574-9fd6fe2d6414 admin admin] PATH_INFO: `/v3/projects/da9411b0cf3a49489a3ffaaf7e716f5a/users/353340e399fb4a498d6c023e9e03a9e8/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:18.508829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 811/1625] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:13:18.516582 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94530b-370d-493d-9403-0ac17b6781ec admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:18.520737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4887f0-2493-46b0-9574-9fd6fe2d6414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9411b0cf3a49489a3ffaaf7e716f5a, user_id=353340e399fb4a498d6c023e9e03a9e8, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:18.524583 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4887f0-2493-46b0-9574-9fd6fe2d6414 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:18.536816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94530b-370d-493d-9403-0ac17b6781ec admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:18.544981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc795961-47d9-4225-848a-e536780df84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:18.546500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc795961-47d9-4225-848a-e536780df84c admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:18.546830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc795961-47d9-4225-848a-e536780df84c admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:18.547090 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc795961-47d9-4225-848a-e536780df84c admin admin] PATH_INFO: `/v3/projects/f0e3e1902b02445c8a328a881d9beecd/users/8d7958c8e947445280f051508f756e5d/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:18.549856 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc795961-47d9-4225-848a-e536780df84c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e3e1902b02445c8a328a881d9beecd, user_id=8d7958c8e947445280f051508f756e5d, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:18.553114 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc795961-47d9-4225-848a-e536780df84c admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:18.567307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 812/1626] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:18 2022] PUT /identity/v3/projects/f0e3e1902b02445c8a328a881d9beecd/users/8d7958c8e947445280f051508f756e5d/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:18.581412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedfae17-bfd2-47e4-b40f-a71a316bb099 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:18.581412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedfae17-bfd2-47e4-b40f-a71a316bb099 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:18.581412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedfae17-bfd2-47e4-b40f-a71a316bb099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:18.620417 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-aedfae17-bfd2-47e4-b40f-a71a316bb099 None None] MFA Rules not processed for user `8d7958c8e947445280f051508f756e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:18.621997 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-aedfae17-bfd2-47e4-b40f-a71a316bb099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:18.673711 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:18.732076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 815/1627] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:18 2022] PUT /identity/v3/projects/da9411b0cf3a49489a3ffaaf7e716f5a/users/353340e399fb4a498d6c023e9e03a9e8/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:18.736402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4887f0-2493-46b0-9574-9fd6fe2d6414 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:18.743818 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 813/1628] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:13:18.756734 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedfae17-bfd2-47e4-b40f-a71a316bb099 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:18.861107 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedfae17-bfd2-47e4-b40f-a71a316bb099 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:18.893713 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4887f0-2493-46b0-9574-9fd6fe2d6414 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:18.913212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfea3cc-6ce0-4954-87a0-4e86431e0dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:18.913212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfea3cc-6ce0-4954-87a0-4e86431e0dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:18.913212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfea3cc-6ce0-4954-87a0-4e86431e0dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:18.915140 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfea3cc-6ce0-4954-87a0-4e86431e0dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:18.916416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfea3cc-6ce0-4954-87a0-4e86431e0dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:18.919328 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfea3cc-6ce0-4954-87a0-4e86431e0dc1 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:18.946342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659daba7-97be-41e7-be86-3e5736c5c4ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:18.946342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659daba7-97be-41e7-be86-3e5736c5c4ca service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:18.946342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659daba7-97be-41e7-be86-3e5736c5c4ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:18.946342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659daba7-97be-41e7-be86-3e5736c5c4ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:18.990263 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 816/1629] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:13:18.991942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfea3cc-6ce0-4954-87a0-4e86431e0dc1 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:19.009270 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfea3cc-6ce0-4954-87a0-4e86431e0dc1 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:19.017773 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006f760-6e32-4da6-aaa9-f4f9c00de526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:19.019190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006f760-6e32-4da6-aaa9-f4f9c00de526 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:19.019705 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006f760-6e32-4da6-aaa9-f4f9c00de526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:19.020238 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006f760-6e32-4da6-aaa9-f4f9c00de526 admin admin] PATH_INFO: `/v3/projects/da9411b0cf3a49489a3ffaaf7e716f5a/users/353340e399fb4a498d6c023e9e03a9e8/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:19.024212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006f760-6e32-4da6-aaa9-f4f9c00de526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9411b0cf3a49489a3ffaaf7e716f5a, user_id=353340e399fb4a498d6c023e9e03a9e8, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:19.029468 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006f760-6e32-4da6-aaa9-f4f9c00de526 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:19.078979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 817/1630] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:18 2022] PUT /identity/v3/projects/da9411b0cf3a49489a3ffaaf7e716f5a/users/353340e399fb4a498d6c023e9e03a9e8/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:19.100391 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25924404-922f-4d03-b988-da0268067e1c None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:19.114069 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25924404-922f-4d03-b988-da0268067e1c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:19.114069 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25924404-922f-4d03-b988-da0268067e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:19.114581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659daba7-97be-41e7-be86-3e5736c5c4ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:19.114581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659daba7-97be-41e7-be86-3e5736c5c4ca service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:19.170431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 814/1631] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:19.172693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-25924404-922f-4d03-b988-da0268067e1c None None] MFA Rules not processed for user `353340e399fb4a498d6c023e9e03a9e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:19.174561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-25924404-922f-4d03-b988-da0268067e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:19.177824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ac53b-1ba9-4a7b-bc56-31d7d063a317 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:19.184801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ac53b-1ba9-4a7b-bc56-31d7d063a317 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:19.185480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ac53b-1ba9-4a7b-bc56-31d7d063a317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:19.284435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-e16ac53b-1ba9-4a7b-bc56-31d7d063a317 None None] MFA Rules not processed for user `a4dfa2bdf4ba49938251c66cc8c9b604`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:19.286207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e16ac53b-1ba9-4a7b-bc56-31d7d063a317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:19.426896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 818/1632] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:13:19.450078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25924404-922f-4d03-b988-da0268067e1c None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:19.541165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:19.544013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 815/1633] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:19 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:19.549374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ac53b-1ba9-4a7b-bc56-31d7d063a317 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:19.646610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25924404-922f-4d03-b988-da0268067e1c None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:19.659792 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074590f-f960-416b-839f-266069fd6d52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:19.667584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074590f-f960-416b-839f-266069fd6d52 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:19.668471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074590f-f960-416b-839f-266069fd6d52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:19.669114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074590f-f960-416b-839f-266069fd6d52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:19.696243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074590f-f960-416b-839f-266069fd6d52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:20.015839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074590f-f960-416b-839f-266069fd6d52 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:20.015839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 819/1634] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:13:20.015839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074590f-f960-416b-839f-266069fd6d52 service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:20.015839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074590f-f960-416b-839f-266069fd6d52 service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:20.017520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5469edb-6655-4996-ae8b-cd885a2975f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:20.017520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5469edb-6655-4996-ae8b-cd885a2975f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:20.017520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5469edb-6655-4996-ae8b-cd885a2975f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:20.019062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ac53b-1ba9-4a7b-bc56-31d7d063a317 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:20.019062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57933235-6bcd-4b90-aa8b-bfb8982ea6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:20.019062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57933235-6bcd-4b90-aa8b-bfb8982ea6ee admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:20.019062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57933235-6bcd-4b90-aa8b-bfb8982ea6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:20.019884 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57933235-6bcd-4b90-aa8b-bfb8982ea6ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:20.020971 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5469edb-6655-4996-ae8b-cd885a2975f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:20.022806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57933235-6bcd-4b90-aa8b-bfb8982ea6ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:20.022806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57933235-6bcd-4b90-aa8b-bfb8982ea6ee admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:20.022806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:20.022806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 816/1635] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:19 2022] POST /identity/v3/projects => generated 403 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:20.022806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57933235-6bcd-4b90-aa8b-bfb8982ea6ee admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:20.023914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5469edb-6655-4996-ae8b-cd885a2975f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:20.023914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5469edb-6655-4996-ae8b-cd885a2975f1 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:20.037055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 820/1636] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:20.047097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5469edb-6655-4996-ae8b-cd885a2975f1 service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:20.094242 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5469edb-6655-4996-ae8b-cd885a2975f1 service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:20.223280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57933235-6bcd-4b90-aa8b-bfb8982ea6ee admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:20.234191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09619721-770a-4cae-9e4b-2925aa67766a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:20.234501 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09619721-770a-4cae-9e4b-2925aa67766a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:20.234739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09619721-770a-4cae-9e4b-2925aa67766a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:20.235173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09619721-770a-4cae-9e4b-2925aa67766a admin admin] PATH_INFO: `/v3/users/a807005d762c4989900ef5fadad37670` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:20.237474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09619721-770a-4cae-9e4b-2925aa67766a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a807005d762c4989900ef5fadad37670)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:20.238368 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: WARNING py.warnings [None req-09619721-770a-4cae-9e4b-2925aa67766a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 15:13:20.238368 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: warnings.warn(msg) Jul 18 15:13:20.238368 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: Jul 18 15:13:20.240476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09619721-770a-4cae-9e4b-2925aa67766a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:20.252922 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e9346-d669-493e-8834-e21b7c678c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:20.256912 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e9346-d669-493e-8834-e21b7c678c5a admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:20.257362 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e9346-d669-493e-8834-e21b7c678c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:20.258046 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e9346-d669-493e-8834-e21b7c678c5a admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:20.259278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e9346-d669-493e-8834-e21b7c678c5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:20.262411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e9346-d669-493e-8834-e21b7c678c5a admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:20.504486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 821/1637] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:20 2022] POST /identity/v3/users => generated 308 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:20.510326 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e9346-d669-493e-8834-e21b7c678c5a admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:20.534115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e9346-d669-493e-8834-e21b7c678c5a admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:20.540576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff9ab4-f2d0-4227-9fc7-d6f17d919286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:20.541593 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff9ab4-f2d0-4227-9fc7-d6f17d919286 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:20.541885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff9ab4-f2d0-4227-9fc7-d6f17d919286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:20.542163 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff9ab4-f2d0-4227-9fc7-d6f17d919286 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:20.543118 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff9ab4-f2d0-4227-9fc7-d6f17d919286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:20.544831 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff9ab4-f2d0-4227-9fc7-d6f17d919286 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:20.586597 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 822/1638] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:20.588614 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff9ab4-f2d0-4227-9fc7-d6f17d919286 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:20.602727 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff9ab4-f2d0-4227-9fc7-d6f17d919286 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:20.609241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f737995-b553-4735-bb84-a205b6b41e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:20.610374 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f737995-b553-4735-bb84-a205b6b41e81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:20.610784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f737995-b553-4735-bb84-a205b6b41e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:20.611198 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f737995-b553-4735-bb84-a205b6b41e81 admin admin] PATH_INFO: `/v3/projects/3c95f5f15fdd47bb811edc213b483cfe/users/f909f7476f9840a0984fc4ce3023d008/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:20.631775 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f737995-b553-4735-bb84-a205b6b41e81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c95f5f15fdd47bb811edc213b483cfe, user_id=f909f7476f9840a0984fc4ce3023d008, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:20.635826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f737995-b553-4735-bb84-a205b6b41e81 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:20.730250 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-09619721-770a-4cae-9e4b-2925aa67766a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a807005d762c4989900ef5fadad37670'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:20.735829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-09619721-770a-4cae-9e4b-2925aa67766a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a807005d762c4989900ef5fadad37670'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:20.737266 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-09619721-770a-4cae-9e4b-2925aa67766a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a807005d762c4989900ef5fadad37670'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:20.837859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 823/1639] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:20 2022] PUT /identity/v3/projects/3c95f5f15fdd47bb811edc213b483cfe/users/f909f7476f9840a0984fc4ce3023d008/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:20.845137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f737995-b553-4735-bb84-a205b6b41e81 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:20.848991 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-8f737995-b553-4735-bb84-a205b6b41e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:20.894023 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-09619721-770a-4cae-9e4b-2925aa67766a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a807005d762c4989900ef5fadad37670'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:20.907108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:20.997238 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-09619721-770a-4cae-9e4b-2925aa67766a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a807005d762c4989900ef5fadad37670'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:21.051332 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:21.066088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 817/1640] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:13:19 2022] DELETE /identity/v3/users/a807005d762c4989900ef5fadad37670 => generated 0 bytes in 1083 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:21.068215 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c72cac-21c1-4068-b971-d103f588ecf6 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:21.068846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c72cac-21c1-4068-b971-d103f588ecf6 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:21.069348 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c72cac-21c1-4068-b971-d103f588ecf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:21.101220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d8c72cac-21c1-4068-b971-d103f588ecf6 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:21.105377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d8c72cac-21c1-4068-b971-d103f588ecf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:21.184017 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f737995-b553-4735-bb84-a205b6b41e81 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:21.202368 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 818/1641] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:21.210653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e3196-4d88-46fb-8d6b-1f523f95d85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:21.213095 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e3196-4d88-46fb-8d6b-1f523f95d85e admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:21.213534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e3196-4d88-46fb-8d6b-1f523f95d85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:21.214729 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e3196-4d88-46fb-8d6b-1f523f95d85e admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:21.215822 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e3196-4d88-46fb-8d6b-1f523f95d85e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:21.218815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e3196-4d88-46fb-8d6b-1f523f95d85e admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:21.238315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 824/1642] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:13:21.248629 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c72cac-21c1-4068-b971-d103f588ecf6 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:21.261682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e3196-4d88-46fb-8d6b-1f523f95d85e admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:21.266385 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-2e5e3196-4d88-46fb-8d6b-1f523f95d85e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:21.267345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c72cac-21c1-4068-b971-d103f588ecf6 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:21.280870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b821a85-5645-4836-8622-86b3cc34bfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:21.282450 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b821a85-5645-4836-8622-86b3cc34bfda admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:21.282997 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b821a85-5645-4836-8622-86b3cc34bfda admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:21.283560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b821a85-5645-4836-8622-86b3cc34bfda admin admin] PATH_INFO: `/v3/projects/3c95f5f15fdd47bb811edc213b483cfe/users/f909f7476f9840a0984fc4ce3023d008/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:21.287749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b821a85-5645-4836-8622-86b3cc34bfda admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c95f5f15fdd47bb811edc213b483cfe, user_id=f909f7476f9840a0984fc4ce3023d008, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:21.294194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b821a85-5645-4836-8622-86b3cc34bfda admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:21.313972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 819/1643] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:21 2022] PUT /identity/v3/projects/3c95f5f15fdd47bb811edc213b483cfe/users/f909f7476f9840a0984fc4ce3023d008/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:21.382367 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e3196-4d88-46fb-8d6b-1f523f95d85e admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:21.577234 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a1a65-939d-4ef0-943a-df4362e21ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:21.578323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a1a65-939d-4ef0-943a-df4362e21ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:21.583140 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a1a65-939d-4ef0-943a-df4362e21ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:21.583985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a1a65-939d-4ef0-943a-df4362e21ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:21.797440 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a1a65-939d-4ef0-943a-df4362e21ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:21.806944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a1a65-939d-4ef0-943a-df4362e21ba6 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:21.831850 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 825/1644] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:22.570165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba474ad-be85-494f-9dd6-55cfb8d1c03b None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:22.572087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba474ad-be85-494f-9dd6-55cfb8d1c03b None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:22.572087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba474ad-be85-494f-9dd6-55cfb8d1c03b None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:22.576425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 820/1645] 10.222.0.83 () {54 vars in 962 bytes} [Mon Jul 18 15:13:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:13:22.592888 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3f259-dd32-412c-ae70-d7f2d26c2cc6 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:22.594193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3f259-dd32-412c-ae70-d7f2d26c2cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:22.594754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3f259-dd32-412c-ae70-d7f2d26c2cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:22.659296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-85e3f259-dd32-412c-ae70-d7f2d26c2cc6 None None] MFA Rules not processed for user `7e1355d18fb049468651b3675c217d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:22.661189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-85e3f259-dd32-412c-ae70-d7f2d26c2cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:22.813588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 826/1646] 10.222.0.83 () {58 vars in 1059 bytes} [Mon Jul 18 15:13:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:22.815679 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f87c6-04c7-4b85-860a-3561fa7c3b8e None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:22.816360 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f87c6-04c7-4b85-860a-3561fa7c3b8e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:22.816679 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f87c6-04c7-4b85-860a-3561fa7c3b8e None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:22.819082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 821/1647] 10.222.0.83 () {54 vars in 912 bytes} [Mon Jul 18 15:13:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 15:13:22.824791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:22.843512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128462a-b9f6-4b9a-8248-bf3d68a144d3 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:22.843512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128462a-b9f6-4b9a-8248-bf3d68a144d3 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:22.843512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128462a-b9f6-4b9a-8248-bf3d68a144d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:22.974904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-0128462a-b9f6-4b9a-8248-bf3d68a144d3 None None] MFA Rules not processed for user `7e1355d18fb049468651b3675c217d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:22.993848 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-0128462a-b9f6-4b9a-8248-bf3d68a144d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:23.022335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 827/1648] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Jul 18 15:13:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:23.041219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f87c6-04c7-4b85-860a-3561fa7c3b8e None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:23.046779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2b8f87c6-04c7-4b85-860a-3561fa7c3b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:23.389134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f87c6-04c7-4b85-860a-3561fa7c3b8e None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:23.399708 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c704629-898a-4480-840b-2dd71cb01da4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:23.401718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c704629-898a-4480-840b-2dd71cb01da4 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:23.401718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c704629-898a-4480-840b-2dd71cb01da4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:23.401718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c704629-898a-4480-840b-2dd71cb01da4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:23.443450 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c704629-898a-4480-840b-2dd71cb01da4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:23.445074 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c704629-898a-4480-840b-2dd71cb01da4 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:23.478241 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 822/1649] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:24.062845 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128462a-b9f6-4b9a-8248-bf3d68a144d3 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:24.064685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-0128462a-b9f6-4b9a-8248-bf3d68a144d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:24.112211 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128462a-b9f6-4b9a-8248-bf3d68a144d3 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:24.128733 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01b15e-3da2-413f-be0f-af540cf34b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:24.131008 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01b15e-3da2-413f-be0f-af540cf34b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:24.134177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01b15e-3da2-413f-be0f-af540cf34b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:24.135730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01b15e-3da2-413f-be0f-af540cf34b18 admin admin] PATH_INFO: `/v3/projects/8058c29fd4e04cf9a4e29dc603ce9a38` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:24.140494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01b15e-3da2-413f-be0f-af540cf34b18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8058c29fd4e04cf9a4e29dc603ce9a38)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:24.144695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: WARNING py.warnings [None req-ef01b15e-3da2-413f-be0f-af540cf34b18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 15:13:24.144695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: warnings.warn(msg) Jul 18 15:13:24.144695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: Jul 18 15:13:24.149910 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01b15e-3da2-413f-be0f-af540cf34b18 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:24.396146 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ef01b15e-3da2-413f-be0f-af540cf34b18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8058c29fd4e04cf9a4e29dc603ce9a38', 'request_id': 'req-ef01b15e-3da2-413f-be0f-af540cf34b18'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:24.424141 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ef01b15e-3da2-413f-be0f-af540cf34b18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8058c29fd4e04cf9a4e29dc603ce9a38', 'request_id': 'req-ef01b15e-3da2-413f-be0f-af540cf34b18'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:24.559142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ef01b15e-3da2-413f-be0f-af540cf34b18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8058c29fd4e04cf9a4e29dc603ce9a38', 'request_id': 'req-ef01b15e-3da2-413f-be0f-af540cf34b18'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:24.588646 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ef01b15e-3da2-413f-be0f-af540cf34b18 admin admin] The token cache is being invalidate because project 8058c29fd4e04cf9a4e29dc603ce9a38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:13:24.589569 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ef01b15e-3da2-413f-be0f-af540cf34b18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:24.591458 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ef01b15e-3da2-413f-be0f-af540cf34b18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:24.596769 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 828/1650] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:13:24 2022] DELETE /identity/v3/projects/8058c29fd4e04cf9a4e29dc603ce9a38 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:24.601015 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c704629-898a-4480-840b-2dd71cb01da4 service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:24.613759 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c704629-898a-4480-840b-2dd71cb01da4 service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:24.687914 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a9fb6-3d75-4fce-88f0-248123e8c2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:24.689391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a9fb6-3d75-4fce-88f0-248123e8c2c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:24.689873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a9fb6-3d75-4fce-88f0-248123e8c2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:24.690290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a9fb6-3d75-4fce-88f0-248123e8c2c8 admin admin] PATH_INFO: `/v3/users/353340e399fb4a498d6c023e9e03a9e8` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:24.692012 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a9fb6-3d75-4fce-88f0-248123e8c2c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=353340e399fb4a498d6c023e9e03a9e8)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:24.696570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a9fb6-3d75-4fce-88f0-248123e8c2c8 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:24.700485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:25.081617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ec2a9fb6-3d75-4fce-88f0-248123e8c2c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '353340e399fb4a498d6c023e9e03a9e8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:25.082576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ec2a9fb6-3d75-4fce-88f0-248123e8c2c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '353340e399fb4a498d6c023e9e03a9e8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:25.083752 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ec2a9fb6-3d75-4fce-88f0-248123e8c2c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '353340e399fb4a498d6c023e9e03a9e8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:25.193727 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ec2a9fb6-3d75-4fce-88f0-248123e8c2c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '353340e399fb4a498d6c023e9e03a9e8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:25.235941 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ec2a9fb6-3d75-4fce-88f0-248123e8c2c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '353340e399fb4a498d6c023e9e03a9e8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:25.272994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 823/1651] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:13:24 2022] DELETE /identity/v3/users/353340e399fb4a498d6c023e9e03a9e8 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:26.733591 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21679af-2d85-4592-a620-517542439630 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:26.734724 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21679af-2d85-4592-a620-517542439630 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:26.735711 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21679af-2d85-4592-a620-517542439630 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:26.737691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 829/1652] 10.222.0.83 () {54 vars in 962 bytes} [Mon Jul 18 15:13:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:13:26.749139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d66b8-5848-4858-822f-b660da641183 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:26.749754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d66b8-5848-4858-822f-b660da641183 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:26.749754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d66b8-5848-4858-822f-b660da641183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:26.797498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-457d66b8-5848-4858-822f-b660da641183 None None] MFA Rules not processed for user `7e1355d18fb049468651b3675c217d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:26.798461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-457d66b8-5848-4858-822f-b660da641183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:26.980349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 824/1653] 10.222.0.83 () {58 vars in 1059 bytes} [Mon Jul 18 15:13:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:13:26.990973 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56130e5-8e9e-4af9-b8ba-35fe6122eb5d None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:26.991356 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56130e5-8e9e-4af9-b8ba-35fe6122eb5d None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:26.991736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56130e5-8e9e-4af9-b8ba-35fe6122eb5d None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:26.994891 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 830/1654] 10.222.0.83 () {54 vars in 912 bytes} [Mon Jul 18 15:13:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:13:27.004048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51fdf80-ee53-481c-96ff-f2a03b847b4d None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:27.004408 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51fdf80-ee53-481c-96ff-f2a03b847b4d None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:27.004911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51fdf80-ee53-481c-96ff-f2a03b847b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:27.060864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d51fdf80-ee53-481c-96ff-f2a03b847b4d None None] MFA Rules not processed for user `7e1355d18fb049468651b3675c217d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:27.063794 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d51fdf80-ee53-481c-96ff-f2a03b847b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:27.101187 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 825/1655] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Jul 18 15:13:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:27.129860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56130e5-8e9e-4af9-b8ba-35fe6122eb5d None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:27.134691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c56130e5-8e9e-4af9-b8ba-35fe6122eb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:27.312664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56130e5-8e9e-4af9-b8ba-35fe6122eb5d None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:27.326093 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44287426-1fbf-4292-9666-bf0a4a413e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:27.326820 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44287426-1fbf-4292-9666-bf0a4a413e12 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:27.327113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44287426-1fbf-4292-9666-bf0a4a413e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:27.327401 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44287426-1fbf-4292-9666-bf0a4a413e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:27.367512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44287426-1fbf-4292-9666-bf0a4a413e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:27.369010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44287426-1fbf-4292-9666-bf0a4a413e12 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:27.397890 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 831/1656] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:13:27.816643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1984634-d379-4fdc-8466-d26b2967b2a6 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:27.819019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1984634-d379-4fdc-8466-d26b2967b2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:27.821326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1984634-d379-4fdc-8466-d26b2967b2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:27.872068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-a1984634-d379-4fdc-8466-d26b2967b2a6 None None] MFA Rules not processed for user `f909f7476f9840a0984fc4ce3023d008`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:27.873779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a1984634-d379-4fdc-8466-d26b2967b2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:28.067365 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44287426-1fbf-4292-9666-bf0a4a413e12 service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:28.068414 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:28.073766 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-44287426-1fbf-4292-9666-bf0a4a413e12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:28.159286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 826/1657] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:27 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:28.220460 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:28.481809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44287426-1fbf-4292-9666-bf0a4a413e12 service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:28.505840 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9d6e8-36dd-4502-a42b-6a8a231e7169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:28.508867 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9d6e8-36dd-4502-a42b-6a8a231e7169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:28.510014 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9d6e8-36dd-4502-a42b-6a8a231e7169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:28.510972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9d6e8-36dd-4502-a42b-6a8a231e7169 admin admin] PATH_INFO: `/v3/projects/da9411b0cf3a49489a3ffaaf7e716f5a` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:28.520545 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9d6e8-36dd-4502-a42b-6a8a231e7169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da9411b0cf3a49489a3ffaaf7e716f5a)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:28.525448 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9d6e8-36dd-4502-a42b-6a8a231e7169 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:28.786654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-78a9d6e8-36dd-4502-a42b-6a8a231e7169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da9411b0cf3a49489a3ffaaf7e716f5a', 'request_id': 'req-78a9d6e8-36dd-4502-a42b-6a8a231e7169'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:28.815753 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-78a9d6e8-36dd-4502-a42b-6a8a231e7169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da9411b0cf3a49489a3ffaaf7e716f5a', 'request_id': 'req-78a9d6e8-36dd-4502-a42b-6a8a231e7169'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:29.033741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-78a9d6e8-36dd-4502-a42b-6a8a231e7169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da9411b0cf3a49489a3ffaaf7e716f5a', 'request_id': 'req-78a9d6e8-36dd-4502-a42b-6a8a231e7169'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:29.137510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-78a9d6e8-36dd-4502-a42b-6a8a231e7169 admin admin] The token cache is being invalidate because project da9411b0cf3a49489a3ffaaf7e716f5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:13:29.138182 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-78a9d6e8-36dd-4502-a42b-6a8a231e7169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:29.142640 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-78a9d6e8-36dd-4502-a42b-6a8a231e7169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:29.159212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 832/1658] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:13:28 2022] DELETE /identity/v3/projects/da9411b0cf3a49489a3ffaaf7e716f5a => generated 0 bytes in 1090 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:29.161239 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82f140-41a6-4580-ac6f-406b94e67f9f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:29.161665 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82f140-41a6-4580-ac6f-406b94e67f9f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:29.164781 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82f140-41a6-4580-ac6f-406b94e67f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:29.256084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-0c82f140-41a6-4580-ac6f-406b94e67f9f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:29.260750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0c82f140-41a6-4580-ac6f-406b94e67f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:29.393518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 827/1659] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:29.401985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251b2c7-815c-40c5-9a59-8e4740d518b4 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:29.402257 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251b2c7-815c-40c5-9a59-8e4740d518b4 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:29.402708 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251b2c7-815c-40c5-9a59-8e4740d518b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:29.435766 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-1251b2c7-815c-40c5-9a59-8e4740d518b4 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:29.437084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1251b2c7-815c-40c5-9a59-8e4740d518b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:29.461604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 833/1660] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:13:29.465002 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82f140-41a6-4580-ac6f-406b94e67f9f None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:29.497340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82f140-41a6-4580-ac6f-406b94e67f9f None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:29.514466 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869a856-28e7-4137-8f17-2da64a124cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:29.515597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869a856-28e7-4137-8f17-2da64a124cda admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:29.516172 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869a856-28e7-4137-8f17-2da64a124cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:29.518205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869a856-28e7-4137-8f17-2da64a124cda admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:29.519289 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869a856-28e7-4137-8f17-2da64a124cda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:13:29.519786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869a856-28e7-4137-8f17-2da64a124cda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:29.524289 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869a856-28e7-4137-8f17-2da64a124cda admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:29.563045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 828/1661] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:13:29.563791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251b2c7-815c-40c5-9a59-8e4740d518b4 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:29.589564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251b2c7-815c-40c5-9a59-8e4740d518b4 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:29.618275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f2fd2-21a4-48cd-ac27-1d9bf6eb11a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:29.623986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f2fd2-21a4-48cd-ac27-1d9bf6eb11a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:29.624097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f2fd2-21a4-48cd-ac27-1d9bf6eb11a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:29.624379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f2fd2-21a4-48cd-ac27-1d9bf6eb11a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:29.625558 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f2fd2-21a4-48cd-ac27-1d9bf6eb11a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:29.638681 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f2fd2-21a4-48cd-ac27-1d9bf6eb11a0 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:29.797623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 834/1662] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:29 2022] POST /identity/v3/projects => generated 409 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:13:29.802232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869a856-28e7-4137-8f17-2da64a124cda admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:29.816963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869a856-28e7-4137-8f17-2da64a124cda admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:29.967640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a53faf-0e70-45c8-b8ee-82090893734b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:29.968720 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a53faf-0e70-45c8-b8ee-82090893734b admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:29.969211 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a53faf-0e70-45c8-b8ee-82090893734b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:29.969688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a53faf-0e70-45c8-b8ee-82090893734b admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:29.971110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a53faf-0e70-45c8-b8ee-82090893734b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:29.973780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a53faf-0e70-45c8-b8ee-82090893734b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:30.111368 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 829/1663] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:29 2022] POST /identity/v3/users => generated 305 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:30.116250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f2fd2-21a4-48cd-ac27-1d9bf6eb11a0 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:30.143124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f2fd2-21a4-48cd-ac27-1d9bf6eb11a0 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:30.157015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e3560-2443-48d6-be97-37b6385225ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:30.157731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e3560-2443-48d6-be97-37b6385225ea admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:30.158253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e3560-2443-48d6-be97-37b6385225ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:30.158736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e3560-2443-48d6-be97-37b6385225ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:30.159492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e3560-2443-48d6-be97-37b6385225ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:30.166106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e3560-2443-48d6-be97-37b6385225ea admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:30.196408 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 835/1664] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:30.198096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a53faf-0e70-45c8-b8ee-82090893734b admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:30.248286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a53faf-0e70-45c8-b8ee-82090893734b admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:30.270733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1467f2-ea72-45a6-b27f-c7a84d753aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:30.271436 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1467f2-ea72-45a6-b27f-c7a84d753aac admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:30.271725 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1467f2-ea72-45a6-b27f-c7a84d753aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:30.272209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1467f2-ea72-45a6-b27f-c7a84d753aac admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:30.277886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1467f2-ea72-45a6-b27f-c7a84d753aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:30.281286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1467f2-ea72-45a6-b27f-c7a84d753aac admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:30.313957 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 830/1665] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:13:30.318187 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e3560-2443-48d6-be97-37b6385225ea admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:30.331929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e3560-2443-48d6-be97-37b6385225ea admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:30.352896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d2b8b-7226-4a0f-9033-ab23d3d5cce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:30.353768 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d2b8b-7226-4a0f-9033-ab23d3d5cce5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:30.356770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d2b8b-7226-4a0f-9033-ab23d3d5cce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:30.357070 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d2b8b-7226-4a0f-9033-ab23d3d5cce5 admin admin] PATH_INFO: `/v3/projects/e826d7d80f2540e8931657120df02b72/users/e5a1c653e2dd469bbf4ad61e71cf2dc0/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:30.395897 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d2b8b-7226-4a0f-9033-ab23d3d5cce5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e826d7d80f2540e8931657120df02b72, user_id=e5a1c653e2dd469bbf4ad61e71cf2dc0, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:30.404968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d2b8b-7226-4a0f-9033-ab23d3d5cce5 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:30.536466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 836/1666] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:30 2022] PUT /identity/v3/projects/e826d7d80f2540e8931657120df02b72/users/e5a1c653e2dd469bbf4ad61e71cf2dc0/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:30.595505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c84411-f846-427f-9204-78a8019d0a2c None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:30.595505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c84411-f846-427f-9204-78a8019d0a2c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:30.595505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c84411-f846-427f-9204-78a8019d0a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:30.643973 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-07c84411-f846-427f-9204-78a8019d0a2c None None] MFA Rules not processed for user `e5a1c653e2dd469bbf4ad61e71cf2dc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:30.646342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-07c84411-f846-427f-9204-78a8019d0a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:30.823801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:30.902719 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 831/1667] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:30 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:30.902921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d2b8b-7226-4a0f-9033-ab23d3d5cce5 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:31.025211 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff7648-05dd-4b53-9305-c15a3408d30a None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:31.037527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff7648-05dd-4b53-9305-c15a3408d30a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:31.037527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff7648-05dd-4b53-9305-c15a3408d30a None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:31.037527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 832/1668] 10.222.0.83 () {54 vars in 921 bytes} [Mon Jul 18 15:13:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:13:31.078002 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:31.111678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d2b8b-7226-4a0f-9033-ab23d3d5cce5 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:31.121527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5909f20-6fbd-4b9e-b190-aefc244f0571 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:31.122521 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5909f20-6fbd-4b9e-b190-aefc244f0571 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:31.123300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5909f20-6fbd-4b9e-b190-aefc244f0571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:31.140569 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609be4d-c75c-49d4-94ae-1235fdec0fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:31.141275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609be4d-c75c-49d4-94ae-1235fdec0fca admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:31.141552 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609be4d-c75c-49d4-94ae-1235fdec0fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:31.141830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609be4d-c75c-49d4-94ae-1235fdec0fca admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:31.142576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609be4d-c75c-49d4-94ae-1235fdec0fca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:31.150742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609be4d-c75c-49d4-94ae-1235fdec0fca admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:31.179943 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-f5909f20-6fbd-4b9e-b190-aefc244f0571 None None] MFA Rules not processed for user `76cc0ae48b5a49c6a7a1c11a08c389e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:31.183583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-f5909f20-6fbd-4b9e-b190-aefc244f0571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:31.316636 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 837/1669] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:30 2022] POST /identity/v3/projects => generated 407 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:31.321155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609be4d-c75c-49d4-94ae-1235fdec0fca admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:31.431846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 833/1670] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Jul 18 15:13:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:13:31.565614 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609be4d-c75c-49d4-94ae-1235fdec0fca admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:31.575803 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae3ede-c509-4da8-ab8d-47154752ea07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:31.576639 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae3ede-c509-4da8-ab8d-47154752ea07 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:31.577038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae3ede-c509-4da8-ab8d-47154752ea07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:31.577434 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae3ede-c509-4da8-ab8d-47154752ea07 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:31.579061 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae3ede-c509-4da8-ab8d-47154752ea07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:31.587876 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae3ede-c509-4da8-ab8d-47154752ea07 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:31.727177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 838/1671] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:31 2022] POST /identity/v3/users => generated 310 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:31.733116 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5909f20-6fbd-4b9e-b190-aefc244f0571 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:31.738769 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d0f21-9416-406b-92c6-d1f4f6bf3c2f None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:31.739323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d0f21-9416-406b-92c6-d1f4f6bf3c2f None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:31.739750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d0f21-9416-406b-92c6-d1f4f6bf3c2f None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:31.755966 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 839/1672] 10.222.0.83 () {54 vars in 961 bytes} [Mon Jul 18 15:13:31 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:13:31.764485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5909f20-6fbd-4b9e-b190-aefc244f0571 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:31.766158 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85235c-e08c-4da6-af4a-a320ae005aea None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:31.770196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85235c-e08c-4da6-af4a-a320ae005aea None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:31.775358 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85235c-e08c-4da6-af4a-a320ae005aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:31.791687 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b878275-f75f-4ce8-abae-9054715e17fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:31.801982 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b878275-f75f-4ce8-abae-9054715e17fb admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:31.803614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b878275-f75f-4ce8-abae-9054715e17fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:31.807147 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b878275-f75f-4ce8-abae-9054715e17fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:31.817902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b878275-f75f-4ce8-abae-9054715e17fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:31.847286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b878275-f75f-4ce8-abae-9054715e17fb admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:31.889189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-ff85235c-e08c-4da6-af4a-a320ae005aea None None] MFA Rules not processed for user `76cc0ae48b5a49c6a7a1c11a08c389e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:31.909229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 834/1673] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:31.913865 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b878275-f75f-4ce8-abae-9054715e17fb admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:31.914819 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ff85235c-e08c-4da6-af4a-a320ae005aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:31.994158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b878275-f75f-4ce8-abae-9054715e17fb admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:32.019391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a23062-7064-42d3-9ce2-2ec0f39d17c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:32.020308 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a23062-7064-42d3-9ce2-2ec0f39d17c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:32.020597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a23062-7064-42d3-9ce2-2ec0f39d17c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:32.020875 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a23062-7064-42d3-9ce2-2ec0f39d17c1 admin admin] PATH_INFO: `/v3/projects/591112c245214d9eb9cde5df54feebdb/users/86fe3352ce7a42f0b2bdac7184e643d5/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:32.091365 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a23062-7064-42d3-9ce2-2ec0f39d17c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=591112c245214d9eb9cde5df54feebdb, user_id=86fe3352ce7a42f0b2bdac7184e643d5, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:32.094991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a23062-7064-42d3-9ce2-2ec0f39d17c1 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:32.186295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 840/1674] 10.222.0.83 () {58 vars in 1058 bytes} [Mon Jul 18 15:13:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:32.200509 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85235c-e08c-4da6-af4a-a320ae005aea None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:32.236072 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 835/1675] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:31 2022] PUT /identity/v3/projects/591112c245214d9eb9cde5df54feebdb/users/86fe3352ce7a42f0b2bdac7184e643d5/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:32.237783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a23062-7064-42d3-9ce2-2ec0f39d17c1 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:32.246992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85235c-e08c-4da6-af4a-a320ae005aea None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:32.404662 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a23062-7064-42d3-9ce2-2ec0f39d17c1 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:32.417782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b09ff-f20b-4bb5-9017-5be4552c3138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:32.421113 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b09ff-f20b-4bb5-9017-5be4552c3138 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:32.421113 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b09ff-f20b-4bb5-9017-5be4552c3138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:32.421113 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b09ff-f20b-4bb5-9017-5be4552c3138 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:32.423674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b09ff-f20b-4bb5-9017-5be4552c3138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:32.431110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b09ff-f20b-4bb5-9017-5be4552c3138 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:32.459157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 836/1676] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:32.465974 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b09ff-f20b-4bb5-9017-5be4552c3138 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:32.496753 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b09ff-f20b-4bb5-9017-5be4552c3138 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:32.502659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c009f-88dd-46bf-abfc-77050b60a79f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76cc0ae48b5a49c6a7a1c11a08c389e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:32.504276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c009f-88dd-46bf-abfc-77050b60a79f service placement] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:32.505954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c009f-88dd-46bf-abfc-77050b60a79f service placement] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:32.522024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c009f-88dd-46bf-abfc-77050b60a79f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:32.525336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fab9c-f80b-465e-84fb-09f9f459ad5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:32.526505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fab9c-f80b-465e-84fb-09f9f459ad5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:32.527002 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fab9c-f80b-465e-84fb-09f9f459ad5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:32.527440 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fab9c-f80b-465e-84fb-09f9f459ad5a admin admin] PATH_INFO: `/v3/projects/591112c245214d9eb9cde5df54feebdb/users/86fe3352ce7a42f0b2bdac7184e643d5/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:32.529912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fab9c-f80b-465e-84fb-09f9f459ad5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=591112c245214d9eb9cde5df54feebdb, user_id=86fe3352ce7a42f0b2bdac7184e643d5, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:32.532903 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fab9c-f80b-465e-84fb-09f9f459ad5a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:32.545665 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:32.559365 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c009f-88dd-46bf-abfc-77050b60a79f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:32.565529 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c009f-88dd-46bf-abfc-77050b60a79f service placement] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:32.588241 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 837/1677] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:32 2022] PUT /identity/v3/projects/591112c245214d9eb9cde5df54feebdb/users/86fe3352ce7a42f0b2bdac7184e643d5/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:32.594221 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 841/1678] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:13:32.600897 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e57ed8-d64d-4d92-b290-e7b1d645f172 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:32.601205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e57ed8-d64d-4d92-b290-e7b1d645f172 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:32.601458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e57ed8-d64d-4d92-b290-e7b1d645f172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:32.640479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-34e57ed8-d64d-4d92-b290-e7b1d645f172 None None] MFA Rules not processed for user `86fe3352ce7a42f0b2bdac7184e643d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:32.642589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-34e57ed8-d64d-4d92-b290-e7b1d645f172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:32.668876 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:32.883842 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 838/1679] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:32 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:32.947076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c009f-88dd-46bf-abfc-77050b60a79f service placement] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:32.949275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-fe6c009f-88dd-46bf-abfc-77050b60a79f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:33.271818 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c009f-88dd-46bf-abfc-77050b60a79f service placement] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:33.431016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a086f5d-74c7-418f-8db1-9f15b5b4d75f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:33.432963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a086f5d-74c7-418f-8db1-9f15b5b4d75f service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:33.434159 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a086f5d-74c7-418f-8db1-9f15b5b4d75f service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:33.437199 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a086f5d-74c7-418f-8db1-9f15b5b4d75f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:33.472970 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a086f5d-74c7-418f-8db1-9f15b5b4d75f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:33.480198 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a086f5d-74c7-418f-8db1-9f15b5b4d75f service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:33.523700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 842/1680] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:32 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:34.573898 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e57ed8-d64d-4d92-b290-e7b1d645f172 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:34.593223 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e57ed8-d64d-4d92-b290-e7b1d645f172 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:34.599989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3f16a-5339-4135-8693-1fc35dc5041e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:34.607921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3f16a-5339-4135-8693-1fc35dc5041e service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:34.607921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3f16a-5339-4135-8693-1fc35dc5041e service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:34.607921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3f16a-5339-4135-8693-1fc35dc5041e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:34.713991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3f16a-5339-4135-8693-1fc35dc5041e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:34.716958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3f16a-5339-4135-8693-1fc35dc5041e service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:34.754841 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 839/1681] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:34 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:36.547490 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97c24a-8b9f-4cba-9cd0-9c56077204c5 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:36.548163 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97c24a-8b9f-4cba-9cd0-9c56077204c5 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:36.548329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97c24a-8b9f-4cba-9cd0-9c56077204c5 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:36.550273 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 843/1682] 10.222.0.83 () {54 vars in 921 bytes} [Mon Jul 18 15:13:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:13:36.582823 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a36456-0595-482c-b6b7-fe726d82a795 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:36.582823 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a36456-0595-482c-b6b7-fe726d82a795 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:36.582823 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a36456-0595-482c-b6b7-fe726d82a795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:36.643456 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97c24a-8b9f-4cba-9cd0-9c56077204c5 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:36.645417 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ac97c24a-8b9f-4cba-9cd0-9c56077204c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:36.656457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-75a36456-0595-482c-b6b7-fe726d82a795 None None] MFA Rules not processed for user `76cc0ae48b5a49c6a7a1c11a08c389e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:36.665473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-75a36456-0595-482c-b6b7-fe726d82a795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:36.827071 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97c24a-8b9f-4cba-9cd0-9c56077204c5 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:36.837629 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07992463-231e-4f83-96fc-295f2954a264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:36.838086 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07992463-231e-4f83-96fc-295f2954a264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:36.839575 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07992463-231e-4f83-96fc-295f2954a264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:36.840952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07992463-231e-4f83-96fc-295f2954a264 admin admin] PATH_INFO: `/v3/users/36d2ee1e945d481a946336506846f88b` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:36.858219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07992463-231e-4f83-96fc-295f2954a264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36d2ee1e945d481a946336506846f88b)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:36.860901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07992463-231e-4f83-96fc-295f2954a264 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:36.962870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 840/1683] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Jul 18 15:13:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:36.972583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4884b0e-726e-4d8b-a951-f87d790c5c4b None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:36.972583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4884b0e-726e-4d8b-a951-f87d790c5c4b None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:36.972583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4884b0e-726e-4d8b-a951-f87d790c5c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:37.104960 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-f4884b0e-726e-4d8b-a951-f87d790c5c4b None None] MFA Rules not processed for user `22b86eef7820462fa4e6cbb22f30b853`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:37.106500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-f4884b0e-726e-4d8b-a951-f87d790c5c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:37.224494 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:37.282019 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-07992463-231e-4f83-96fc-295f2954a264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36d2ee1e945d481a946336506846f88b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:37.291123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:37.308314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-07992463-231e-4f83-96fc-295f2954a264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36d2ee1e945d481a946336506846f88b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:37.309098 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-07992463-231e-4f83-96fc-295f2954a264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36d2ee1e945d481a946336506846f88b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:37.323059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 841/1684] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:37.324768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4884b0e-726e-4d8b-a951-f87d790c5c4b None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:37.332854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-f4884b0e-726e-4d8b-a951-f87d790c5c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:37.371115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4884b0e-726e-4d8b-a951-f87d790c5c4b None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:37.371591 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-07992463-231e-4f83-96fc-295f2954a264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36d2ee1e945d481a946336506846f88b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:37.394189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309fb6d-47ad-4c22-aa45-1ed591befd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:37.399776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309fb6d-47ad-4c22-aa45-1ed591befd1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:37.405241 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309fb6d-47ad-4c22-aa45-1ed591befd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:37.406046 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309fb6d-47ad-4c22-aa45-1ed591befd1b admin admin] PATH_INFO: `/v3/users/a4dfa2bdf4ba49938251c66cc8c9b604` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:37.410155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309fb6d-47ad-4c22-aa45-1ed591befd1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4dfa2bdf4ba49938251c66cc8c9b604)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:37.412386 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309fb6d-47ad-4c22-aa45-1ed591befd1b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:37.455632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-07992463-231e-4f83-96fc-295f2954a264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36d2ee1e945d481a946336506846f88b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:37.470407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 844/1685] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:13:36 2022] DELETE /identity/v3/users/36d2ee1e945d481a946336506846f88b => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:37.470407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07992463-231e-4f83-96fc-295f2954a264 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:37.471513 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-07992463-231e-4f83-96fc-295f2954a264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:37.668502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07992463-231e-4f83-96fc-295f2954a264 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:37.687128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36956840-6274-43ef-a0e1-abefb3f64eb6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76cc0ae48b5a49c6a7a1c11a08c389e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:37.688085 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36956840-6274-43ef-a0e1-abefb3f64eb6 service placement] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:37.688551 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36956840-6274-43ef-a0e1-abefb3f64eb6 service placement] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:37.689013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36956840-6274-43ef-a0e1-abefb3f64eb6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:37.699989 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-36956840-6274-43ef-a0e1-abefb3f64eb6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:37.723745 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36956840-6274-43ef-a0e1-abefb3f64eb6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:37.732448 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36956840-6274-43ef-a0e1-abefb3f64eb6 service placement] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:37.756491 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 845/1686] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:37.758420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36956840-6274-43ef-a0e1-abefb3f64eb6 service placement] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:37.770285 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-36956840-6274-43ef-a0e1-abefb3f64eb6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:37.841059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5309fb6d-47ad-4c22-aa45-1ed591befd1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4dfa2bdf4ba49938251c66cc8c9b604'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:37.847010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5309fb6d-47ad-4c22-aa45-1ed591befd1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4dfa2bdf4ba49938251c66cc8c9b604'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:37.847728 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5309fb6d-47ad-4c22-aa45-1ed591befd1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4dfa2bdf4ba49938251c66cc8c9b604'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:37.929420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36956840-6274-43ef-a0e1-abefb3f64eb6 service placement] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:37.938438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d0413-1fb6-463b-ac85-10904d250d78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:37.939482 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d0413-1fb6-463b-ac85-10904d250d78 service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:37.939949 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d0413-1fb6-463b-ac85-10904d250d78 service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:37.940446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d0413-1fb6-463b-ac85-10904d250d78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:37.944403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-895d0413-1fb6-463b-ac85-10904d250d78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:37.998457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5309fb6d-47ad-4c22-aa45-1ed591befd1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4dfa2bdf4ba49938251c66cc8c9b604'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:38.101939 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5309fb6d-47ad-4c22-aa45-1ed591befd1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4dfa2bdf4ba49938251c66cc8c9b604'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:38.154937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d0413-1fb6-463b-ac85-10904d250d78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:38.157060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d0413-1fb6-463b-ac85-10904d250d78 service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:38.208210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 842/1688] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:13:37 2022] DELETE /identity/v3/users/a4dfa2bdf4ba49938251c66cc8c9b604 => generated 0 bytes in 884 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:38.208581 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 846/1688] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:38.210133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3e8f2-7ece-402f-91d3-dbfda69812ca None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:38.210743 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3e8f2-7ece-402f-91d3-dbfda69812ca None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:38.211203 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3e8f2-7ece-402f-91d3-dbfda69812ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:38.211913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309fb6d-47ad-4c22-aa45-1ed591befd1b admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:38.235301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-5309fb6d-47ad-4c22-aa45-1ed591befd1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:38.246445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-40a3e8f2-7ece-402f-91d3-dbfda69812ca None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:38.252199 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-40a3e8f2-7ece-402f-91d3-dbfda69812ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:38.361208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 847/1689] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:38.462719 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3e8f2-7ece-402f-91d3-dbfda69812ca None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:38.465019 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-40a3e8f2-7ece-402f-91d3-dbfda69812ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:38.531619 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:38.545437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309fb6d-47ad-4c22-aa45-1ed591befd1b admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:38.571809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469dac5-1e02-4126-8d85-0f5189cfcd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:38.572735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469dac5-1e02-4126-8d85-0f5189cfcd72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:38.573021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469dac5-1e02-4126-8d85-0f5189cfcd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:38.573304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469dac5-1e02-4126-8d85-0f5189cfcd72 admin admin] PATH_INFO: `/v3/users/eca68707420847969407a77dbb6bc67c` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:38.581342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469dac5-1e02-4126-8d85-0f5189cfcd72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca68707420847969407a77dbb6bc67c)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:38.583797 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469dac5-1e02-4126-8d85-0f5189cfcd72 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:38.773790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3e8f2-7ece-402f-91d3-dbfda69812ca None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:38.795148 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16774af7-4f54-4269-a0d2-26d38b5f0b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:38.798101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16774af7-4f54-4269-a0d2-26d38b5f0b7d service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:38.802184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16774af7-4f54-4269-a0d2-26d38b5f0b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:38.802184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16774af7-4f54-4269-a0d2-26d38b5f0b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:38.879178 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16774af7-4f54-4269-a0d2-26d38b5f0b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:38.896081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16774af7-4f54-4269-a0d2-26d38b5f0b7d service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:38.901503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5469dac5-1e02-4126-8d85-0f5189cfcd72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca68707420847969407a77dbb6bc67c'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:38.902935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5469dac5-1e02-4126-8d85-0f5189cfcd72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca68707420847969407a77dbb6bc67c'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:38.906765 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5469dac5-1e02-4126-8d85-0f5189cfcd72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca68707420847969407a77dbb6bc67c'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:38.959943 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 848/1690] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:13:39.067487 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5469dac5-1e02-4126-8d85-0f5189cfcd72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca68707420847969407a77dbb6bc67c'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:39.071130 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:39.109088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5469dac5-1e02-4126-8d85-0f5189cfcd72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca68707420847969407a77dbb6bc67c'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:39.160325 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 843/1691] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:13:38 2022] DELETE /identity/v3/users/eca68707420847969407a77dbb6bc67c => generated 0 bytes in 951 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:39.176494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107bdd3-c3bb-486e-bc97-9617a06318e7 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:39.176494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107bdd3-c3bb-486e-bc97-9617a06318e7 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:39.176494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107bdd3-c3bb-486e-bc97-9617a06318e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:39.239701 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-8107bdd3-c3bb-486e-bc97-9617a06318e7 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:39.246224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-8107bdd3-c3bb-486e-bc97-9617a06318e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:39.563918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 849/1692] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:39.673499 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469dac5-1e02-4126-8d85-0f5189cfcd72 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:39.682729 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-5469dac5-1e02-4126-8d85-0f5189cfcd72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:40.081448 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469dac5-1e02-4126-8d85-0f5189cfcd72 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:40.105596 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85339a-f4f7-4c0a-a6ba-c37c65cb98da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:40.105596 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85339a-f4f7-4c0a-a6ba-c37c65cb98da service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:40.105931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85339a-f4f7-4c0a-a6ba-c37c65cb98da service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:40.106032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85339a-f4f7-4c0a-a6ba-c37c65cb98da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:40.128034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85339a-f4f7-4c0a-a6ba-c37c65cb98da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:40.134646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85339a-f4f7-4c0a-a6ba-c37c65cb98da service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:40.169850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 844/1693] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:13:41.038507 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd45b4b-1438-4e15-8139-3b3d8000f34a None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:41.039675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd45b4b-1438-4e15-8139-3b3d8000f34a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:41.040334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd45b4b-1438-4e15-8139-3b3d8000f34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:41.125802 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-7cd45b4b-1438-4e15-8139-3b3d8000f34a None None] MFA Rules not processed for user `648c869550ee437a8e775e997c741e58`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:41.128687 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7cd45b4b-1438-4e15-8139-3b3d8000f34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:41.208125 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:41.281783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ce3b8-ec5c-4548-b3a1-d5d463e0b576 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:41.283055 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ce3b8-ec5c-4548-b3a1-d5d463e0b576 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:41.283559 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ce3b8-ec5c-4548-b3a1-d5d463e0b576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:41.347370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 850/1694] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:41.360205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246200b-6172-4f09-bc85-13a196e53763 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:41.361310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246200b-6172-4f09-bc85-13a196e53763 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:41.362166 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246200b-6172-4f09-bc85-13a196e53763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:41.421931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-7c4ce3b8-ec5c-4548-b3a1-d5d463e0b576 None None] MFA Rules not processed for user `f3f0c37e426640f9960c1da232605e55`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:41.429511 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7c4ce3b8-ec5c-4548-b3a1-d5d463e0b576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:41.448209 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-1246200b-6172-4f09-bc85-13a196e53763 None None] MFA Rules not processed for user `648c869550ee437a8e775e997c741e58`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:41.451039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1246200b-6172-4f09-bc85-13a196e53763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:41.503866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 851/1695] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:41.534412 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246200b-6172-4f09-bc85-13a196e53763 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:41.539206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1246200b-6172-4f09-bc85-13a196e53763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:41.652684 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 845/1696] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:41.654199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ce3b8-ec5c-4548-b3a1-d5d463e0b576 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:41.663015 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7c4ce3b8-ec5c-4548-b3a1-d5d463e0b576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:41.707732 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ce3b8-ec5c-4548-b3a1-d5d463e0b576 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:41.729351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127c7b0-b09e-43d2-b3c6-853647553856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:41.731123 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127c7b0-b09e-43d2-b3c6-853647553856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:41.735219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127c7b0-b09e-43d2-b3c6-853647553856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:41.735219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127c7b0-b09e-43d2-b3c6-853647553856 admin admin] PATH_INFO: `/v3/projects/c9e531c14bb746679c26077041def6ab` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:41.735219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127c7b0-b09e-43d2-b3c6-853647553856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9e531c14bb746679c26077041def6ab)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:41.736784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127c7b0-b09e-43d2-b3c6-853647553856 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:41.921916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-d127c7b0-b09e-43d2-b3c6-853647553856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9e531c14bb746679c26077041def6ab', 'request_id': 'req-d127c7b0-b09e-43d2-b3c6-853647553856'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:41.936859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-d127c7b0-b09e-43d2-b3c6-853647553856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9e531c14bb746679c26077041def6ab', 'request_id': 'req-d127c7b0-b09e-43d2-b3c6-853647553856'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:41.949314 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:41.953274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-d127c7b0-b09e-43d2-b3c6-853647553856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9e531c14bb746679c26077041def6ab', 'request_id': 'req-d127c7b0-b09e-43d2-b3c6-853647553856'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:42.022440 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246200b-6172-4f09-bc85-13a196e53763 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:42.069231 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-d127c7b0-b09e-43d2-b3c6-853647553856 admin admin] The token cache is being invalidate because project c9e531c14bb746679c26077041def6ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:13:42.069641 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-d127c7b0-b09e-43d2-b3c6-853647553856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:42.075267 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-d127c7b0-b09e-43d2-b3c6-853647553856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:42.078922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 846/1697] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:13:41 2022] DELETE /identity/v3/projects/c9e531c14bb746679c26077041def6ab => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:42.080079 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127c7b0-b09e-43d2-b3c6-853647553856 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:42.089955 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d127c7b0-b09e-43d2-b3c6-853647553856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:42.292283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06318059-4e81-45c3-9fe8-4a2750be5a9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:42.294477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06318059-4e81-45c3-9fe8-4a2750be5a9a service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:42.295038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06318059-4e81-45c3-9fe8-4a2750be5a9a service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:42.295235 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06318059-4e81-45c3-9fe8-4a2750be5a9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:42.297416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-06318059-4e81-45c3-9fe8-4a2750be5a9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:42.383448 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127c7b0-b09e-43d2-b3c6-853647553856 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:42.392753 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5103ac-a85c-4a65-a58b-960bf1dadc06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:42.393753 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5103ac-a85c-4a65-a58b-960bf1dadc06 service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:42.394232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5103ac-a85c-4a65-a58b-960bf1dadc06 service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:42.394859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5103ac-a85c-4a65-a58b-960bf1dadc06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:42.401041 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-be5103ac-a85c-4a65-a58b-960bf1dadc06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:42.413663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:42.417020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06318059-4e81-45c3-9fe8-4a2750be5a9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:42.419770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06318059-4e81-45c3-9fe8-4a2750be5a9a service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:42.432996 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 852/1698] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 900 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:42.435271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06318059-4e81-45c3-9fe8-4a2750be5a9a service nova] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:42.435730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-06318059-4e81-45c3-9fe8-4a2750be5a9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:42.599442 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06318059-4e81-45c3-9fe8-4a2750be5a9a service nova] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:42.624705 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73964aa4-2d55-4b23-95b7-3668b0f1109b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:42.626456 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73964aa4-2d55-4b23-95b7-3668b0f1109b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:42.627514 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73964aa4-2d55-4b23-95b7-3668b0f1109b admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:42.628021 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73964aa4-2d55-4b23-95b7-3668b0f1109b admin admin] PATH_INFO: `/v3/users/f909f7476f9840a0984fc4ce3023d008` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:42.634067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73964aa4-2d55-4b23-95b7-3668b0f1109b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f909f7476f9840a0984fc4ce3023d008)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:42.638207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73964aa4-2d55-4b23-95b7-3668b0f1109b admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:42.646256 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5103ac-a85c-4a65-a58b-960bf1dadc06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:42.648894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5103ac-a85c-4a65-a58b-960bf1dadc06 service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:42.660859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 847/1699] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:42.661778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5103ac-a85c-4a65-a58b-960bf1dadc06 service nova] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:42.664941 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-be5103ac-a85c-4a65-a58b-960bf1dadc06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:42.682866 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5103ac-a85c-4a65-a58b-960bf1dadc06 service nova] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:42.688977 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:42.689884 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:42.690199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:42.690483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b admin admin] PATH_INFO: `/v3/projects/7d9e87e7a63642338bfa2ccc3110f27c` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:42.691911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d9e87e7a63642338bfa2ccc3110f27c)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:42.696003 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:43.035139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d9e87e7a63642338bfa2ccc3110f27c', 'request_id': 'req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:43.048043 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-73964aa4-2d55-4b23-95b7-3668b0f1109b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f909f7476f9840a0984fc4ce3023d008'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:43.074152 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-73964aa4-2d55-4b23-95b7-3668b0f1109b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f909f7476f9840a0984fc4ce3023d008'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:43.074614 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-73964aa4-2d55-4b23-95b7-3668b0f1109b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f909f7476f9840a0984fc4ce3023d008'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:43.075491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d9e87e7a63642338bfa2ccc3110f27c', 'request_id': 'req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:43.110173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d9e87e7a63642338bfa2ccc3110f27c', 'request_id': 'req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:43.119635 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-73964aa4-2d55-4b23-95b7-3668b0f1109b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f909f7476f9840a0984fc4ce3023d008'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:43.231980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-73964aa4-2d55-4b23-95b7-3668b0f1109b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f909f7476f9840a0984fc4ce3023d008'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:43.241302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 853/1700] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:13:42 2022] DELETE /identity/v3/users/f909f7476f9840a0984fc4ce3023d008 => generated 0 bytes in 807 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:43.247392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab35550-8e91-401b-992c-5d6995944adf None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:43.247392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab35550-8e91-401b-992c-5d6995944adf None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:43.255770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab35550-8e91-401b-992c-5d6995944adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:43.326615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-3ab35550-8e91-401b-992c-5d6995944adf None None] MFA Rules not processed for user `21de90974572419e906e7ac5d9d8292d`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:43.328296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3ab35550-8e91-401b-992c-5d6995944adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:43.339263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b admin admin] The token cache is being invalidate because project 7d9e87e7a63642338bfa2ccc3110f27c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:13:43.340023 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:43.341560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:43.374173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 848/1701] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:13:42 2022] DELETE /identity/v3/projects/7d9e87e7a63642338bfa2ccc3110f27c => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:43.376556 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:43.383085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:43.417869 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:43.533886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 854/1702] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:43.534984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab35550-8e91-401b-992c-5d6995944adf None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:43.542951 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3ab35550-8e91-401b-992c-5d6995944adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:43.562518 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:43.696362 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1266e4a-9a13-49f9-9070-50ec4a26ba6b admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:43.779942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab35550-8e91-401b-992c-5d6995944adf None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:43.807387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5774ed-c6c6-4565-8f58-42be46f3d26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:43.808152 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5774ed-c6c6-4565-8f58-42be46f3d26f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:43.808855 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5774ed-c6c6-4565-8f58-42be46f3d26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:43.809789 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5774ed-c6c6-4565-8f58-42be46f3d26f admin admin] PATH_INFO: `/v3/users/8d7958c8e947445280f051508f756e5d` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:43.811846 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5774ed-c6c6-4565-8f58-42be46f3d26f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d7958c8e947445280f051508f756e5d)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:43.825992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5774ed-c6c6-4565-8f58-42be46f3d26f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:43.908922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81b35f-d8fd-4405-84df-e8a2c944708d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:43.910081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81b35f-d8fd-4405-84df-e8a2c944708d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:43.910374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81b35f-d8fd-4405-84df-e8a2c944708d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:43.910696 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81b35f-d8fd-4405-84df-e8a2c944708d admin admin] PATH_INFO: `/v3/projects/c5023c967d9841d29cc721efe9684b89` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:43.912205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81b35f-d8fd-4405-84df-e8a2c944708d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5023c967d9841d29cc721efe9684b89)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:43.914571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81b35f-d8fd-4405-84df-e8a2c944708d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:44.196319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0b81b35f-d8fd-4405-84df-e8a2c944708d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5023c967d9841d29cc721efe9684b89', 'request_id': 'req-0b81b35f-d8fd-4405-84df-e8a2c944708d'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:44.214209 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4f5774ed-c6c6-4565-8f58-42be46f3d26f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d7958c8e947445280f051508f756e5d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:44.219768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0b81b35f-d8fd-4405-84df-e8a2c944708d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5023c967d9841d29cc721efe9684b89', 'request_id': 'req-0b81b35f-d8fd-4405-84df-e8a2c944708d'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:44.248302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0b81b35f-d8fd-4405-84df-e8a2c944708d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5023c967d9841d29cc721efe9684b89', 'request_id': 'req-0b81b35f-d8fd-4405-84df-e8a2c944708d'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:44.248937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4f5774ed-c6c6-4565-8f58-42be46f3d26f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d7958c8e947445280f051508f756e5d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:44.250561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4f5774ed-c6c6-4565-8f58-42be46f3d26f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d7958c8e947445280f051508f756e5d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:44.298000 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4f5774ed-c6c6-4565-8f58-42be46f3d26f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d7958c8e947445280f051508f756e5d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:44.381720 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0b81b35f-d8fd-4405-84df-e8a2c944708d admin admin] The token cache is being invalidate because project c5023c967d9841d29cc721efe9684b89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:13:44.391742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0b81b35f-d8fd-4405-84df-e8a2c944708d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:44.397638 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0b81b35f-d8fd-4405-84df-e8a2c944708d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:44.401126 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 849/1703] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:13:43 2022] DELETE /identity/v3/projects/c5023c967d9841d29cc721efe9684b89 => generated 0 bytes in 1026 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:44.410950 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be81011-bfba-45dc-ad3e-d22d81b11a15 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:44.412029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be81011-bfba-45dc-ad3e-d22d81b11a15 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:44.412944 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be81011-bfba-45dc-ad3e-d22d81b11a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:44.469046 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4f5774ed-c6c6-4565-8f58-42be46f3d26f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d7958c8e947445280f051508f756e5d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:44.482411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 855/1704] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:13:43 2022] DELETE /identity/v3/users/8d7958c8e947445280f051508f756e5d => generated 0 bytes in 946 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:44.485845 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073f8d1-a73b-4b79-ba93-9835b6befd09 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:44.486191 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073f8d1-a73b-4b79-ba93-9835b6befd09 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:44.486481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073f8d1-a73b-4b79-ba93-9835b6befd09 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:44.493189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 856/1705] 10.222.0.83 () {54 vars in 912 bytes} [Mon Jul 18 15:13:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:13:44.502895 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3019fa-45ab-4351-b6a1-1f84524de0bf None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:44.502895 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3019fa-45ab-4351-b6a1-1f84524de0bf None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:44.502895 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3019fa-45ab-4351-b6a1-1f84524de0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:44.503171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-1be81011-bfba-45dc-ad3e-d22d81b11a15 None None] MFA Rules not processed for user `21de90974572419e906e7ac5d9d8292d`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:44.507351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-1be81011-bfba-45dc-ad3e-d22d81b11a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:44.560408 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-8e3019fa-45ab-4351-b6a1-1f84524de0bf None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:44.564827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-8e3019fa-45ab-4351-b6a1-1f84524de0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:44.683537 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:44.729827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 850/1706] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:44.732633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be81011-bfba-45dc-ad3e-d22d81b11a15 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:44.736319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-1be81011-bfba-45dc-ad3e-d22d81b11a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:44.775339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 857/1707] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:44.777233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f052ac-439e-4404-9d64-f392eb80fca5 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:44.777412 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f052ac-439e-4404-9d64-f392eb80fca5 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:44.777619 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f052ac-439e-4404-9d64-f392eb80fca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:44.870671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-31f052ac-439e-4404-9d64-f392eb80fca5 None None] MFA Rules not processed for user `fd86979e0f0d4a4e9921ee5704901f74`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:44.871853 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-31f052ac-439e-4404-9d64-f392eb80fca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:44.889260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:44.958632 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be81011-bfba-45dc-ad3e-d22d81b11a15 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:44.969334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e7e72-6f67-4e53-9d9e-cb1ef2dedbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:44.970709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e7e72-6f67-4e53-9d9e-cb1ef2dedbfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:44.971184 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e7e72-6f67-4e53-9d9e-cb1ef2dedbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:44.971480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e7e72-6f67-4e53-9d9e-cb1ef2dedbfa admin admin] PATH_INFO: `/v3/projects/3c95f5f15fdd47bb811edc213b483cfe` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:44.973680 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e7e72-6f67-4e53-9d9e-cb1ef2dedbfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c95f5f15fdd47bb811edc213b483cfe)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:44.978242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e7e72-6f67-4e53-9d9e-cb1ef2dedbfa admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:45.036197 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 858/1708] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Jul 18 15:13:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:45.041999 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f052ac-439e-4404-9d64-f392eb80fca5 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:45.047312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-31f052ac-439e-4404-9d64-f392eb80fca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:45.129237 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-508e7e72-6f67-4e53-9d9e-cb1ef2dedbfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c95f5f15fdd47bb811edc213b483cfe', 'request_id': 'req-508e7e72-6f67-4e53-9d9e-cb1ef2dedbfa'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:45.152911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-508e7e72-6f67-4e53-9d9e-cb1ef2dedbfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c95f5f15fdd47bb811edc213b483cfe', 'request_id': 'req-508e7e72-6f67-4e53-9d9e-cb1ef2dedbfa'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:45.173457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-508e7e72-6f67-4e53-9d9e-cb1ef2dedbfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c95f5f15fdd47bb811edc213b483cfe', 'request_id': 'req-508e7e72-6f67-4e53-9d9e-cb1ef2dedbfa'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:45.268508 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f052ac-439e-4404-9d64-f392eb80fca5 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:45.282362 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-508e7e72-6f67-4e53-9d9e-cb1ef2dedbfa admin admin] The token cache is being invalidate because project 3c95f5f15fdd47bb811edc213b483cfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:13:45.282953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-508e7e72-6f67-4e53-9d9e-cb1ef2dedbfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:45.283915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-508e7e72-6f67-4e53-9d9e-cb1ef2dedbfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:45.291929 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 851/1709] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:13:44 2022] DELETE /identity/v3/projects/3c95f5f15fdd47bb811edc213b483cfe => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:45.293898 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999d45d-820e-49c4-ad61-48591546ae7f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:45.298781 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999d45d-820e-49c4-ad61-48591546ae7f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:45.299365 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999d45d-820e-49c4-ad61-48591546ae7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:45.350324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b999d45d-820e-49c4-ad61-48591546ae7f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:45.357983 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b999d45d-820e-49c4-ad61-48591546ae7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:45.544283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9be57a-0b75-465c-92fe-459861ffeed4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:45.547752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9be57a-0b75-465c-92fe-459861ffeed4 service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:45.550976 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9be57a-0b75-465c-92fe-459861ffeed4 service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:45.551758 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9be57a-0b75-465c-92fe-459861ffeed4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:45.559802 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-cf9be57a-0b75-465c-92fe-459861ffeed4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:45.619021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 852/1710] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:45.620816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999d45d-820e-49c4-ad61-48591546ae7f None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:45.623292 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b999d45d-820e-49c4-ad61-48591546ae7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:45.710859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:45.832213 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9be57a-0b75-465c-92fe-459861ffeed4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:45.837697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9be57a-0b75-465c-92fe-459861ffeed4 service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:45.857793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:45.876699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 859/1711] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 840 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:45.883783 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea1949-fd5e-4910-b296-57b2fb762ca5 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:45.884260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea1949-fd5e-4910-b296-57b2fb762ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:45.884723 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea1949-fd5e-4910-b296-57b2fb762ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:45.909103 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999d45d-820e-49c4-ad61-48591546ae7f None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:45.952051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e1258-01ea-4d0f-bfdb-b959a813ff89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:45.954274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e1258-01ea-4d0f-bfdb-b959a813ff89 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:45.954274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e1258-01ea-4d0f-bfdb-b959a813ff89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:45.954575 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e1258-01ea-4d0f-bfdb-b959a813ff89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:45.958312 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-658e1258-01ea-4d0f-bfdb-b959a813ff89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:45.978699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-b0ea1949-fd5e-4910-b296-57b2fb762ca5 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:45.984625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b0ea1949-fd5e-4910-b296-57b2fb762ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:46.011262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e1258-01ea-4d0f-bfdb-b959a813ff89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:46.018999 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e1258-01ea-4d0f-bfdb-b959a813ff89 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:46.024149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 860/1712] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:46.025982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea1949-fd5e-4910-b296-57b2fb762ca5 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:46.050472 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 853/1713] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:46.072069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ba407-ea85-4f04-92e2-c28c6cbfb7ec None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:46.072610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ba407-ea85-4f04-92e2-c28c6cbfb7ec None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:46.073042 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ba407-ea85-4f04-92e2-c28c6cbfb7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:46.083022 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea1949-fd5e-4910-b296-57b2fb762ca5 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:46.090518 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e61227-cca4-4899-8eec-8e326ed75ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:46.091489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e61227-cca4-4899-8eec-8e326ed75ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:46.091897 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e61227-cca4-4899-8eec-8e326ed75ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:46.092298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e61227-cca4-4899-8eec-8e326ed75ae6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:46.093263 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e61227-cca4-4899-8eec-8e326ed75ae6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:13:46.093864 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e61227-cca4-4899-8eec-8e326ed75ae6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:46.097133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e61227-cca4-4899-8eec-8e326ed75ae6 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:46.115171 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 861/1714] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:13:46.120422 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e61227-cca4-4899-8eec-8e326ed75ae6 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:46.134822 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e61227-cca4-4899-8eec-8e326ed75ae6 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:46.140878 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd95971-4135-4c65-8794-79a5b16708ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:46.141427 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd95971-4135-4c65-8794-79a5b16708ad admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:46.141720 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd95971-4135-4c65-8794-79a5b16708ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:46.142003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd95971-4135-4c65-8794-79a5b16708ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:46.142896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd95971-4135-4c65-8794-79a5b16708ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:46.145583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-e00ba407-ea85-4f04-92e2-c28c6cbfb7ec None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:46.148470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e00ba407-ea85-4f04-92e2-c28c6cbfb7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:46.148730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd95971-4135-4c65-8794-79a5b16708ad admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:46.195745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 854/1715] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:46.200816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ba407-ea85-4f04-92e2-c28c6cbfb7ec None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:46.231488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ba407-ea85-4f04-92e2-c28c6cbfb7ec None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:46.268280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be730b4e-fa38-4041-aff9-281b8f490909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:46.270458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be730b4e-fa38-4041-aff9-281b8f490909 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:46.271303 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be730b4e-fa38-4041-aff9-281b8f490909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:46.272478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be730b4e-fa38-4041-aff9-281b8f490909 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:46.274640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be730b4e-fa38-4041-aff9-281b8f490909 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:13:46.276404 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be730b4e-fa38-4041-aff9-281b8f490909 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:46.281019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be730b4e-fa38-4041-aff9-281b8f490909 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:46.323551 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 862/1716] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:46 2022] POST /identity/v3/projects => generated 357 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:13:46.329324 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd95971-4135-4c65-8794-79a5b16708ad admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:46.348077 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 855/1717] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 15:13:46.350652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be730b4e-fa38-4041-aff9-281b8f490909 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:46.413588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be730b4e-fa38-4041-aff9-281b8f490909 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:46.559698 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd95971-4135-4c65-8794-79a5b16708ad admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:46.591055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a11547-70a7-4a60-b0c6-624908dd407c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:46.600518 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a11547-70a7-4a60-b0c6-624908dd407c admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:46.602720 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a11547-70a7-4a60-b0c6-624908dd407c admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:46.603810 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a11547-70a7-4a60-b0c6-624908dd407c admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:46.604749 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a11547-70a7-4a60-b0c6-624908dd407c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:46.627324 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a11547-70a7-4a60-b0c6-624908dd407c admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:46.759707 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b0a40-158b-441b-939e-8998a05b44bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:46.764693 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b0a40-158b-441b-939e-8998a05b44bc admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:46.766062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b0a40-158b-441b-939e-8998a05b44bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:46.766627 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b0a40-158b-441b-939e-8998a05b44bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:46.770164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b0a40-158b-441b-939e-8998a05b44bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:46.770919 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 863/1718] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:46 2022] POST /identity/v3/users => generated 279 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:46.772901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a11547-70a7-4a60-b0c6-624908dd407c admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:46.776932 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b0a40-158b-441b-939e-8998a05b44bc admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:46.808032 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a11547-70a7-4a60-b0c6-624908dd407c admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:46.828328 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727df14-21d6-4ddf-afd0-6b84e77c303f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:46.829047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727df14-21d6-4ddf-afd0-6b84e77c303f admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:46.829362 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727df14-21d6-4ddf-afd0-6b84e77c303f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:46.829647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727df14-21d6-4ddf-afd0-6b84e77c303f admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:46.835847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727df14-21d6-4ddf-afd0-6b84e77c303f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:46.844781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727df14-21d6-4ddf-afd0-6b84e77c303f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:46.864535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 864/1719] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:46.870390 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727df14-21d6-4ddf-afd0-6b84e77c303f admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:46.887496 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727df14-21d6-4ddf-afd0-6b84e77c303f admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:46.900481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8938e4-a65c-4ec9-a809-ce9db6bfa807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:46.902281 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8938e4-a65c-4ec9-a809-ce9db6bfa807 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:46.903017 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8938e4-a65c-4ec9-a809-ce9db6bfa807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:46.903808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8938e4-a65c-4ec9-a809-ce9db6bfa807 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:46.905815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8938e4-a65c-4ec9-a809-ce9db6bfa807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:46.909568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8938e4-a65c-4ec9-a809-ce9db6bfa807 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:46.965012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 865/1720] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:46.966916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8938e4-a65c-4ec9-a809-ce9db6bfa807 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:46.985278 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 856/1721] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:46 2022] POST /identity/v3/projects => generated 365 bytes in 637 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:46.993085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b0a40-158b-441b-939e-8998a05b44bc admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:47.032096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8938e4-a65c-4ec9-a809-ce9db6bfa807 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:47.144695 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b0a40-158b-441b-939e-8998a05b44bc admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:47.156093 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9dd24-bb2d-4bf0-823e-67f4d18713e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:47.156806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9dd24-bb2d-4bf0-823e-67f4d18713e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:47.157096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9dd24-bb2d-4bf0-823e-67f4d18713e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:47.157380 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9dd24-bb2d-4bf0-823e-67f4d18713e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:47.158106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e9dd24-bb2d-4bf0-823e-67f4d18713e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:47.160248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e9dd24-bb2d-4bf0-823e-67f4d18713e3 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:47.263666 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 857/1722] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:46 2022] POST /identity/v3/users => generated 283 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:13:47.268994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9dd24-bb2d-4bf0-823e-67f4d18713e3 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:47.271466 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-29e9dd24-bb2d-4bf0-823e-67f4d18713e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:47.304013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9dd24-bb2d-4bf0-823e-67f4d18713e3 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:47.321415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fc2f1-f3f1-44fb-a207-670c6a44b1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:47.322357 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fc2f1-f3f1-44fb-a207-670c6a44b1df admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:47.322678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fc2f1-f3f1-44fb-a207-670c6a44b1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:47.322958 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fc2f1-f3f1-44fb-a207-670c6a44b1df admin admin] PATH_INFO: `/v3/projects/184452ed59424f808208e40117401e90/users/ab4e8ed13a79403e8397840370abc1d9/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:47.333507 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7c7c5-f104-4769-adbd-421eff9834b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:47.334797 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7c7c5-f104-4769-adbd-421eff9834b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:47.335257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7c7c5-f104-4769-adbd-421eff9834b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:47.335563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7c7c5-f104-4769-adbd-421eff9834b4 admin admin] PATH_INFO: `/v3/projects/f0e3e1902b02445c8a328a881d9beecd` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:47.341431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7c7c5-f104-4769-adbd-421eff9834b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0e3e1902b02445c8a328a881d9beecd)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:47.346426 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0fc2f1-f3f1-44fb-a207-670c6a44b1df admin admin] RBAC: Authorizing `identity:create_grant(project_id=184452ed59424f808208e40117401e90, user_id=ab4e8ed13a79403e8397840370abc1d9, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:47.352016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0fc2f1-f3f1-44fb-a207-670c6a44b1df admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:47.355723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7c7c5-f104-4769-adbd-421eff9834b4 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:47.406020 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:47.474126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 866/1723] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:46 2022] PUT /identity/v3/projects/184452ed59424f808208e40117401e90/users/ab4e8ed13a79403e8397840370abc1d9/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:47.485654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fc2f1-f3f1-44fb-a207-670c6a44b1df admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:47.506278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:47.628396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00e7c7c5-f104-4769-adbd-421eff9834b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0e3e1902b02445c8a328a881d9beecd', 'request_id': 'req-00e7c7c5-f104-4769-adbd-421eff9834b4'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:47.648008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00e7c7c5-f104-4769-adbd-421eff9834b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0e3e1902b02445c8a328a881d9beecd', 'request_id': 'req-00e7c7c5-f104-4769-adbd-421eff9834b4'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:47.676283 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00e7c7c5-f104-4769-adbd-421eff9834b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0e3e1902b02445c8a328a881d9beecd', 'request_id': 'req-00e7c7c5-f104-4769-adbd-421eff9834b4'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:47.706764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fc2f1-f3f1-44fb-a207-670c6a44b1df admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:47.793392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00e7c7c5-f104-4769-adbd-421eff9834b4 admin admin] The token cache is being invalidate because project f0e3e1902b02445c8a328a881d9beecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:13:47.794420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00e7c7c5-f104-4769-adbd-421eff9834b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:47.796292 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00e7c7c5-f104-4769-adbd-421eff9834b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:47.801179 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 858/1724] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:13:47 2022] DELETE /identity/v3/projects/f0e3e1902b02445c8a328a881d9beecd => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:47.803504 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ae5dd-da65-4994-b60e-d4f3f3405c9a None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:47.806744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ae5dd-da65-4994-b60e-d4f3f3405c9a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:47.807305 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ae5dd-da65-4994-b60e-d4f3f3405c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:47.903004 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-0e6ae5dd-da65-4994-b60e-d4f3f3405c9a None None] MFA Rules not processed for user `ab4e8ed13a79403e8397840370abc1d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:47.904521 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3ca61-c2fc-4588-a228-4dcea4173792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:47.905439 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3ca61-c2fc-4588-a228-4dcea4173792 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:47.905736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3ca61-c2fc-4588-a228-4dcea4173792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:47.906015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3ca61-c2fc-4588-a228-4dcea4173792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:47.907256 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3ca61-c2fc-4588-a228-4dcea4173792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:47.914136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0e6ae5dd-da65-4994-b60e-d4f3f3405c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:47.921642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3ca61-c2fc-4588-a228-4dcea4173792 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:47.971620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 867/1725] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 494 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:47.977147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da83cd-5656-4072-b575-253882d77290 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:47.977553 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da83cd-5656-4072-b575-253882d77290 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:47.977850 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da83cd-5656-4072-b575-253882d77290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:48.071594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-41da83cd-5656-4072-b575-253882d77290 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:48.075136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-41da83cd-5656-4072-b575-253882d77290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:48.114692 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 868/1726] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:48.122461 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da83cd-5656-4072-b575-253882d77290 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:48.125686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-41da83cd-5656-4072-b575-253882d77290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:48.191811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da83cd-5656-4072-b575-253882d77290 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:48.214247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e53a3-aaf4-4fc2-ad4e-bb380fdc066c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:48.215018 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e53a3-aaf4-4fc2-ad4e-bb380fdc066c admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:48.215315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e53a3-aaf4-4fc2-ad4e-bb380fdc066c admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:48.215611 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e53a3-aaf4-4fc2-ad4e-bb380fdc066c admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:48.216377 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e53a3-aaf4-4fc2-ad4e-bb380fdc066c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:48.218114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e53a3-aaf4-4fc2-ad4e-bb380fdc066c admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:48.231174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 859/1727] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:47 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:48.232628 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41264ae0-ec1e-402b-b69d-1f5d954e512b None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:48.232930 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41264ae0-ec1e-402b-b69d-1f5d954e512b None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:48.233219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41264ae0-ec1e-402b-b69d-1f5d954e512b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:48.262460 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 869/1728] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:48.272824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d95c9-3b7f-46df-a9db-8843727965d8 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:48.273199 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d95c9-3b7f-46df-a9db-8843727965d8 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:48.273490 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d95c9-3b7f-46df-a9db-8843727965d8 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:48.278087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 870/1729] 10.222.0.83 () {54 vars in 912 bytes} [Mon Jul 18 15:13:48 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:13:48.287385 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d95c9-3b7f-46df-a9db-8843727965d8 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:48.291647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-eb6d95c9-3b7f-46df-a9db-8843727965d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:48.302178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-41264ae0-ec1e-402b-b69d-1f5d954e512b None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:48.308327 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-41264ae0-ec1e-402b-b69d-1f5d954e512b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:48.367052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 860/1730] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:48.370022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41264ae0-ec1e-402b-b69d-1f5d954e512b None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:48.381172 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-41264ae0-ec1e-402b-b69d-1f5d954e512b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:48.405575 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41264ae0-ec1e-402b-b69d-1f5d954e512b None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:48.411325 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14d5fb-420c-436c-97e1-5ae93b5b15f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:48.414419 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14d5fb-420c-436c-97e1-5ae93b5b15f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:48.421675 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14d5fb-420c-436c-97e1-5ae93b5b15f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:48.421675 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14d5fb-420c-436c-97e1-5ae93b5b15f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:48.421675 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14d5fb-420c-436c-97e1-5ae93b5b15f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:48.421675 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14d5fb-420c-436c-97e1-5ae93b5b15f1 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:48.536851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d95c9-3b7f-46df-a9db-8843727965d8 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:48.546179 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa47d63-084a-4d56-b0c0-f93e3c800940 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76cc0ae48b5a49c6a7a1c11a08c389e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:48.547270 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa47d63-084a-4d56-b0c0-f93e3c800940 service placement] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:48.547723 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa47d63-084a-4d56-b0c0-f93e3c800940 service placement] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:48.548202 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa47d63-084a-4d56-b0c0-f93e3c800940 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:48.553495 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-baa47d63-084a-4d56-b0c0-f93e3c800940 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:48.597724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 861/1731] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:48 2022] POST /identity/v3/projects => generated 355 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:48.601117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa47d63-084a-4d56-b0c0-f93e3c800940 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:48.601705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14d5fb-420c-436c-97e1-5ae93b5b15f1 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:48.610495 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa47d63-084a-4d56-b0c0-f93e3c800940 service placement] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:48.661682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 871/1732] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:48.664288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0bcba-5cae-4b2a-b522-6bdd22200786 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:48.665804 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0bcba-5cae-4b2a-b522-6bdd22200786 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:48.665804 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0bcba-5cae-4b2a-b522-6bdd22200786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:48.715033 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14d5fb-420c-436c-97e1-5ae93b5b15f1 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:48.719457 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-c7d0bcba-5cae-4b2a-b522-6bdd22200786 None None] MFA Rules not processed for user `fd86979e0f0d4a4e9921ee5704901f74`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:48.722178 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c7d0bcba-5cae-4b2a-b522-6bdd22200786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:48.727750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aab787-6b3b-4715-8c6c-2e9ef526fea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:48.728825 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aab787-6b3b-4715-8c6c-2e9ef526fea0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:48.729008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aab787-6b3b-4715-8c6c-2e9ef526fea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:48.731483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aab787-6b3b-4715-8c6c-2e9ef526fea0 admin admin] PATH_INFO: `/v3/projects/19ae2153d19949f5a9eab06296eaa333/users/dbf97c999c66499bbbc4252159f406b5/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:48.738386 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:48.744636 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:48.772093 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aab787-6b3b-4715-8c6c-2e9ef526fea0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ae2153d19949f5a9eab06296eaa333, user_id=dbf97c999c66499bbbc4252159f406b5, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:48.775219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aab787-6b3b-4715-8c6c-2e9ef526fea0 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:48.863484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 872/1733] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Jul 18 15:13:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:48.864465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0bcba-5cae-4b2a-b522-6bdd22200786 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:48.882516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0bcba-5cae-4b2a-b522-6bdd22200786 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:48.912445 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 862/1734] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:48 2022] PUT /identity/v3/projects/19ae2153d19949f5a9eab06296eaa333/users/dbf97c999c66499bbbc4252159f406b5/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:48.912445 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2d53b-672d-4096-802c-9a4ee115f5c2 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:48.912445 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2d53b-672d-4096-802c-9a4ee115f5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:48.912445 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2d53b-672d-4096-802c-9a4ee115f5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:49.044030 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-43c2d53b-672d-4096-802c-9a4ee115f5c2 None None] MFA Rules not processed for user `22b86eef7820462fa4e6cbb22f30b853`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:49.052149 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-43c2d53b-672d-4096-802c-9a4ee115f5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:49.237189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba3165-b589-4582-8a38-b72814873151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:49.238781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba3165-b589-4582-8a38-b72814873151 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:49.240978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba3165-b589-4582-8a38-b72814873151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:49.241585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba3165-b589-4582-8a38-b72814873151 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:49.243395 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba3165-b589-4582-8a38-b72814873151 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:13:49.244201 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba3165-b589-4582-8a38-b72814873151 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:49.247952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba3165-b589-4582-8a38-b72814873151 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:49.282604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 873/1735] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:13:49.286004 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42469b7-7d14-470c-b007-4dfb51575984 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:49.286891 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42469b7-7d14-470c-b007-4dfb51575984 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:49.287839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42469b7-7d14-470c-b007-4dfb51575984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:49.391458 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-a42469b7-7d14-470c-b007-4dfb51575984 None None] MFA Rules not processed for user `648c869550ee437a8e775e997c741e58`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:49.400724 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a42469b7-7d14-470c-b007-4dfb51575984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:49.418211 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 863/1736] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 516 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:49.419454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2d53b-672d-4096-802c-9a4ee115f5c2 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:49.452084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2d53b-672d-4096-802c-9a4ee115f5c2 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:49.472341 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37f0bd-99fe-40b4-9df8-2a89fb41817d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:49.472341 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37f0bd-99fe-40b4-9df8-2a89fb41817d admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:49.472341 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37f0bd-99fe-40b4-9df8-2a89fb41817d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:49.472341 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37f0bd-99fe-40b4-9df8-2a89fb41817d admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:49.473205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37f0bd-99fe-40b4-9df8-2a89fb41817d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:49.475495 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37f0bd-99fe-40b4-9df8-2a89fb41817d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:49.587005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 864/1737] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:49 2022] POST /identity/v3/users => generated 284 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:49.588127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37f0bd-99fe-40b4-9df8-2a89fb41817d admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:49.590356 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-da37f0bd-99fe-40b4-9df8-2a89fb41817d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:49.613962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37f0bd-99fe-40b4-9df8-2a89fb41817d admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:49.626291 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1c5d5-0a58-4010-8aa4-a84c3851bda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:49.628169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1c5d5-0a58-4010-8aa4-a84c3851bda7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:49.628641 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1c5d5-0a58-4010-8aa4-a84c3851bda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:49.629066 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1c5d5-0a58-4010-8aa4-a84c3851bda7 admin admin] PATH_INFO: `/v3/users/e5a1c653e2dd469bbf4ad61e71cf2dc0` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:49.631606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1c5d5-0a58-4010-8aa4-a84c3851bda7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5a1c653e2dd469bbf4ad61e71cf2dc0)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:49.634415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1c5d5-0a58-4010-8aa4-a84c3851bda7 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:49.647932 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 874/1738] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:49.659103 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42469b7-7d14-470c-b007-4dfb51575984 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:49.668007 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a42469b7-7d14-470c-b007-4dfb51575984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:49.792426 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42469b7-7d14-470c-b007-4dfb51575984 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:49.796880 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571a653-7def-4840-9e9c-9471b795af28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:49.797715 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571a653-7def-4840-9e9c-9471b795af28 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:49.798240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571a653-7def-4840-9e9c-9471b795af28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:49.798813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571a653-7def-4840-9e9c-9471b795af28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:49.816187 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571a653-7def-4840-9e9c-9471b795af28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:49.819265 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571a653-7def-4840-9e9c-9471b795af28 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:49.844098 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 875/1739] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:49.853352 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46e93c-eb82-4d63-937e-44b27efc2461 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:49.853352 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46e93c-eb82-4d63-937e-44b27efc2461 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:49.853352 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46e93c-eb82-4d63-937e-44b27efc2461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:49.916794 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:49.930049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-f2e1c5d5-0a58-4010-8aa4-a84c3851bda7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5a1c653e2dd469bbf4ad61e71cf2dc0'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:49.934320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-f2e1c5d5-0a58-4010-8aa4-a84c3851bda7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5a1c653e2dd469bbf4ad61e71cf2dc0'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:49.934559 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-f2e1c5d5-0a58-4010-8aa4-a84c3851bda7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5a1c653e2dd469bbf4ad61e71cf2dc0'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:49.942306 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-2a46e93c-eb82-4d63-937e-44b27efc2461 None None] MFA Rules not processed for user `dbf97c999c66499bbbc4252159f406b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:49.947992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-2a46e93c-eb82-4d63-937e-44b27efc2461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:50.025867 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-f2e1c5d5-0a58-4010-8aa4-a84c3851bda7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5a1c653e2dd469bbf4ad61e71cf2dc0'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:50.060524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-f2e1c5d5-0a58-4010-8aa4-a84c3851bda7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5a1c653e2dd469bbf4ad61e71cf2dc0'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:50.103270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 865/1740] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:13:49 2022] DELETE /identity/v3/users/e5a1c653e2dd469bbf4ad61e71cf2dc0 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:50.104685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1c5d5-0a58-4010-8aa4-a84c3851bda7 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:50.110392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 876/1741] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:49 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:50.110392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46e93c-eb82-4d63-937e-44b27efc2461 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:50.110392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-2a46e93c-eb82-4d63-937e-44b27efc2461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:50.116560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-f2e1c5d5-0a58-4010-8aa4-a84c3851bda7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:50.137072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46e93c-eb82-4d63-937e-44b27efc2461 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:50.231718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:50.247389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc75cf4-b7e6-4a1c-a4f0-461d0f46cabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:50.248568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc75cf4-b7e6-4a1c-a4f0-461d0f46cabf admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:50.249794 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc75cf4-b7e6-4a1c-a4f0-461d0f46cabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:50.250382 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc75cf4-b7e6-4a1c-a4f0-461d0f46cabf admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:50.251313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc75cf4-b7e6-4a1c-a4f0-461d0f46cabf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:50.253474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc75cf4-b7e6-4a1c-a4f0-461d0f46cabf admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:50.300080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 877/1742] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:50.301862 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1c5d5-0a58-4010-8aa4-a84c3851bda7 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:50.306370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a1ed7-c3e3-4723-af4f-4692e62d651d None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:50.308654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a1ed7-c3e3-4723-af4f-4692e62d651d None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:50.309255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a1ed7-c3e3-4723-af4f-4692e62d651d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:50.316822 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18ce2f-c8c8-42f2-976b-14b575504a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:50.318781 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18ce2f-c8c8-42f2-976b-14b575504a9a admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:50.319177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18ce2f-c8c8-42f2-976b-14b575504a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:50.321726 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18ce2f-c8c8-42f2-976b-14b575504a9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:50.326821 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18ce2f-c8c8-42f2-976b-14b575504a9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:50.326821 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18ce2f-c8c8-42f2-976b-14b575504a9a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:50.372009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-da0a1ed7-c3e3-4723-af4f-4692e62d651d None None] MFA Rules not processed for user `f3f0c37e426640f9960c1da232605e55`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:50.374711 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-da0a1ed7-c3e3-4723-af4f-4692e62d651d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:50.446807 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 866/1743] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:50 2022] POST /identity/v3/projects => generated 387 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:50.452904 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a5ddb-a3fa-48ca-ab3d-26b481061d80 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:50.453455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a5ddb-a3fa-48ca-ab3d-26b481061d80 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:50.453894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a5ddb-a3fa-48ca-ab3d-26b481061d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:50.503056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b13a5ddb-a3fa-48ca-ab3d-26b481061d80 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:50.505147 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b13a5ddb-a3fa-48ca-ab3d-26b481061d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:50.590183 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 878/1744] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:50.591230 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a1ed7-c3e3-4723-af4f-4692e62d651d None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:50.597554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-da0a1ed7-c3e3-4723-af4f-4692e62d651d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:50.636170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a1ed7-c3e3-4723-af4f-4692e62d651d None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:50.674144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 867/1745] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:50.678579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a5ddb-a3fa-48ca-ab3d-26b481061d80 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:50.727096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a5ddb-a3fa-48ca-ab3d-26b481061d80 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:50.735504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:50.749113 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4e91d-7ac8-4a33-afcb-1045e8712873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:50.751709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4e91d-7ac8-4a33-afcb-1045e8712873 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:50.753853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4e91d-7ac8-4a33-afcb-1045e8712873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:50.754265 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4e91d-7ac8-4a33-afcb-1045e8712873 admin admin] PATH_INFO: `/v3/projects/cc64eae755244c80bbde04f6acf3bd2b/users/8f40970abb8d4d109d2368e7fe1cc63b/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:50.789531 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c6ef1-1cea-45bf-978f-79243fff9f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:50.793904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c6ef1-1cea-45bf-978f-79243fff9f72 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:50.793904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c6ef1-1cea-45bf-978f-79243fff9f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:50.796830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c6ef1-1cea-45bf-978f-79243fff9f72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:50.800963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c6ef1-1cea-45bf-978f-79243fff9f72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:50.817410 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c6ef1-1cea-45bf-978f-79243fff9f72 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:50.835618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4e91d-7ac8-4a33-afcb-1045e8712873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc64eae755244c80bbde04f6acf3bd2b, user_id=8f40970abb8d4d109d2368e7fe1cc63b, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:50.842612 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4e91d-7ac8-4a33-afcb-1045e8712873 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:51.016248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 868/1746] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:50 2022] PUT /identity/v3/projects/cc64eae755244c80bbde04f6acf3bd2b/users/8f40970abb8d4d109d2368e7fe1cc63b/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:51.018181 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4e91d-7ac8-4a33-afcb-1045e8712873 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:51.026469 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 879/1747] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:50 2022] POST /identity/v3/projects => generated 361 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:51.040341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c6ef1-1cea-45bf-978f-79243fff9f72 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:51.048642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b97c6ef1-1cea-45bf-978f-79243fff9f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:51.289500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c6ef1-1cea-45bf-978f-79243fff9f72 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:51.308463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d80e2a-df5a-4cdb-8838-ba3916aa32b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:51.308463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d80e2a-df5a-4cdb-8838-ba3916aa32b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:51.316753 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d80e2a-df5a-4cdb-8838-ba3916aa32b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:51.317067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d80e2a-df5a-4cdb-8838-ba3916aa32b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:51.321446 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4e91d-7ac8-4a33-afcb-1045e8712873 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:51.470122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:51.490859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2763d-ae3e-4f02-97eb-0bf0c2f259aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:51.493001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2763d-ae3e-4f02-97eb-0bf0c2f259aa admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:51.493864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2763d-ae3e-4f02-97eb-0bf0c2f259aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:51.494658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2763d-ae3e-4f02-97eb-0bf0c2f259aa admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:51.497803 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2763d-ae3e-4f02-97eb-0bf0c2f259aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:51.503036 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2763d-ae3e-4f02-97eb-0bf0c2f259aa admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:51.573464 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d80e2a-df5a-4cdb-8838-ba3916aa32b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:51.575876 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d80e2a-df5a-4cdb-8838-ba3916aa32b0 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:51.595942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 880/1748] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:51.601455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d80e2a-df5a-4cdb-8838-ba3916aa32b0 service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:51.649553 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d80e2a-df5a-4cdb-8838-ba3916aa32b0 service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:51.670273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 869/1749] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:51 2022] POST /identity/v3/users => generated 294 bytes in 652 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:51.671580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2763d-ae3e-4f02-97eb-0bf0c2f259aa admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:51.676161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb73bd3-b683-4ce0-98ca-47b5fcce4364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:51.677666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb73bd3-b683-4ce0-98ca-47b5fcce4364 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:51.679001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb73bd3-b683-4ce0-98ca-47b5fcce4364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:51.681253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb73bd3-b683-4ce0-98ca-47b5fcce4364 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:51.682580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb73bd3-b683-4ce0-98ca-47b5fcce4364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:51.688297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb73bd3-b683-4ce0-98ca-47b5fcce4364 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:51.701868 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2763d-ae3e-4f02-97eb-0bf0c2f259aa admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:51.720899 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 881/1750] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:51.721256 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446cfac-9517-4fe9-bc71-51afc25bc308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:51.721987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446cfac-9517-4fe9-bc71-51afc25bc308 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:51.722280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446cfac-9517-4fe9-bc71-51afc25bc308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:51.722395 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb73bd3-b683-4ce0-98ca-47b5fcce4364 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:51.722620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446cfac-9517-4fe9-bc71-51afc25bc308 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:51.723360 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446cfac-9517-4fe9-bc71-51afc25bc308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:51.726015 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446cfac-9517-4fe9-bc71-51afc25bc308 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:51.737972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb73bd3-b683-4ce0-98ca-47b5fcce4364 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:51.745014 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa4bc3-25a5-41d3-8c81-d07943a71d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:51.746780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa4bc3-25a5-41d3-8c81-d07943a71d29 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:51.747477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa4bc3-25a5-41d3-8c81-d07943a71d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:51.747904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa4bc3-25a5-41d3-8c81-d07943a71d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:51.748989 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa4bc3-25a5-41d3-8c81-d07943a71d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:51.751474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa4bc3-25a5-41d3-8c81-d07943a71d29 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:51.777897 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 882/1751] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:51.779273 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa4bc3-25a5-41d3-8c81-d07943a71d29 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:51.799150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa4bc3-25a5-41d3-8c81-d07943a71d29 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:51.809615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251f038-892a-408f-9e41-9c57a84cc320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:51.810698 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251f038-892a-408f-9e41-9c57a84cc320 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:51.812327 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251f038-892a-408f-9e41-9c57a84cc320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:51.812578 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251f038-892a-408f-9e41-9c57a84cc320 admin admin] PATH_INFO: `/v3/projects/cc64eae755244c80bbde04f6acf3bd2b/users/8f40970abb8d4d109d2368e7fe1cc63b/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:51.820021 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251f038-892a-408f-9e41-9c57a84cc320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc64eae755244c80bbde04f6acf3bd2b, user_id=8f40970abb8d4d109d2368e7fe1cc63b, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:51.823621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251f038-892a-408f-9e41-9c57a84cc320 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:51.873259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 870/1752] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:51 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:51.875059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446cfac-9517-4fe9-bc71-51afc25bc308 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:51.895208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 883/1753] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:51 2022] PUT /identity/v3/projects/cc64eae755244c80bbde04f6acf3bd2b/users/8f40970abb8d4d109d2368e7fe1cc63b/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:51.907337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251f038-892a-408f-9e41-9c57a84cc320 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:51.914092 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446cfac-9517-4fe9-bc71-51afc25bc308 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:52.107142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:52.187320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a69e92-0c2a-47a6-803d-ff0f88de4029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:52.188047 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a69e92-0c2a-47a6-803d-ff0f88de4029 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:52.188336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a69e92-0c2a-47a6-803d-ff0f88de4029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:52.191775 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251f038-892a-408f-9e41-9c57a84cc320 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:52.193582 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a69e92-0c2a-47a6-803d-ff0f88de4029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:52.194673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a69e92-0c2a-47a6-803d-ff0f88de4029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:52.197637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a69e92-0c2a-47a6-803d-ff0f88de4029 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:52.206625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ae5bf-41dc-4b36-b978-953952e6635a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:52.217691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ae5bf-41dc-4b36-b978-953952e6635a admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:52.217691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ae5bf-41dc-4b36-b978-953952e6635a admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:52.217691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ae5bf-41dc-4b36-b978-953952e6635a admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:52.217691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ae5bf-41dc-4b36-b978-953952e6635a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:52.231512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ae5bf-41dc-4b36-b978-953952e6635a admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:52.260635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 871/1754] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:52.262765 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93ce39-9f2a-48f2-a2a2-2315d88f037a None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:52.263340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93ce39-9f2a-48f2-a2a2-2315d88f037a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:52.263930 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93ce39-9f2a-48f2-a2a2-2315d88f037a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:52.270420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 884/1755] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:52.271642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ae5bf-41dc-4b36-b978-953952e6635a admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:52.299991 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ae5bf-41dc-4b36-b978-953952e6635a admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:52.333082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83a5e0-1bfc-4c71-bde9-b1d4aab6a404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:52.334107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83a5e0-1bfc-4c71-bde9-b1d4aab6a404 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:52.334400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83a5e0-1bfc-4c71-bde9-b1d4aab6a404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:52.339117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83a5e0-1bfc-4c71-bde9-b1d4aab6a404 admin admin] PATH_INFO: `/v3/projects/8b708ec6dc5e41f4b0b16583391049ff/users/58a41d81210b4e408433c92957c010df/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:52.371728 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-4a93ce39-9f2a-48f2-a2a2-2315d88f037a None None] MFA Rules not processed for user `8f40970abb8d4d109d2368e7fe1cc63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:52.373079 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-4a93ce39-9f2a-48f2-a2a2-2315d88f037a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:52.400158 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83a5e0-1bfc-4c71-bde9-b1d4aab6a404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b708ec6dc5e41f4b0b16583391049ff, user_id=58a41d81210b4e408433c92957c010df, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:52.400158 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83a5e0-1bfc-4c71-bde9-b1d4aab6a404 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:52.559647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 885/1756] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:52 2022] PUT /identity/v3/projects/8b708ec6dc5e41f4b0b16583391049ff/users/58a41d81210b4e408433c92957c010df/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:52.560780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83a5e0-1bfc-4c71-bde9-b1d4aab6a404 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:52.640924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 872/1757] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:52 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:52.646491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaec168-0f03-4204-ac7e-a4f9ca1608bf None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:52.646491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaec168-0f03-4204-ac7e-a4f9ca1608bf None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:52.649005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaec168-0f03-4204-ac7e-a4f9ca1608bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:52.775021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-1aaec168-0f03-4204-ac7e-a4f9ca1608bf None None] MFA Rules not processed for user `58a41d81210b4e408433c92957c010df`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:52.775021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-1aaec168-0f03-4204-ac7e-a4f9ca1608bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:52.913120 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83a5e0-1bfc-4c71-bde9-b1d4aab6a404 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:52.950109 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251dbfab-9d8c-4195-8c8c-9a0efc4f30d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:52.957716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251dbfab-9d8c-4195-8c8c-9a0efc4f30d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:52.957931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251dbfab-9d8c-4195-8c8c-9a0efc4f30d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:52.963578 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251dbfab-9d8c-4195-8c8c-9a0efc4f30d5 admin admin] PATH_INFO: `/v3/projects/92f6eaad43d6446e885c8aae4366010a/users/5a7ecd5c59d34ed0ab9507ad3404744e/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:52.988001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:53.032694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251dbfab-9d8c-4195-8c8c-9a0efc4f30d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f6eaad43d6446e885c8aae4366010a, user_id=5a7ecd5c59d34ed0ab9507ad3404744e, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:53.035609 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251dbfab-9d8c-4195-8c8c-9a0efc4f30d5 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:53.135583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 873/1758] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 488 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:53.135583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaec168-0f03-4204-ac7e-a4f9ca1608bf None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:53.135583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-1aaec168-0f03-4204-ac7e-a4f9ca1608bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:53.186906 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 886/1759] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:52 2022] PUT /identity/v3/projects/92f6eaad43d6446e885c8aae4366010a/users/5a7ecd5c59d34ed0ab9507ad3404744e/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:53.187903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251dbfab-9d8c-4195-8c8c-9a0efc4f30d5 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:53.409815 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaec168-0f03-4204-ac7e-a4f9ca1608bf None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:53.555686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88392092-480e-482f-a27e-b94f3d9d3dbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:53.557923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88392092-480e-482f-a27e-b94f3d9d3dbd service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:53.559190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88392092-480e-482f-a27e-b94f3d9d3dbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:53.560500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88392092-480e-482f-a27e-b94f3d9d3dbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:53.595359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251dbfab-9d8c-4195-8c8c-9a0efc4f30d5 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:53.614236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123d49f-9581-49f7-9e55-618760530b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:53.615038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123d49f-9581-49f7-9e55-618760530b67 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:53.615333 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123d49f-9581-49f7-9e55-618760530b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:53.615621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123d49f-9581-49f7-9e55-618760530b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:53.616351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123d49f-9581-49f7-9e55-618760530b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:53.618835 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123d49f-9581-49f7-9e55-618760530b67 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:53.805258 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 887/1760] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:53 2022] POST /identity/v3/projects => generated 389 bytes in 618 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:53.808299 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123d49f-9581-49f7-9e55-618760530b67 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:53.881448 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:53.902268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88392092-480e-482f-a27e-b94f3d9d3dbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:53.903809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88392092-480e-482f-a27e-b94f3d9d3dbd service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:53.936044 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 874/1761] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:53 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 806 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:53.939205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88392092-480e-482f-a27e-b94f3d9d3dbd service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:53.945910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-88392092-480e-482f-a27e-b94f3d9d3dbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:53.999466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123d49f-9581-49f7-9e55-618760530b67 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:54.007767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168808c-720b-4aab-a60f-7e4b04d713bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:54.012883 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168808c-720b-4aab-a60f-7e4b04d713bd admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:54.013345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168808c-720b-4aab-a60f-7e4b04d713bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:54.013736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168808c-720b-4aab-a60f-7e4b04d713bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:54.015900 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168808c-720b-4aab-a60f-7e4b04d713bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:54.017748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168808c-720b-4aab-a60f-7e4b04d713bd admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:54.027721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88392092-480e-482f-a27e-b94f3d9d3dbd service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:54.042339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 888/1762] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:54.043316 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168808c-720b-4aab-a60f-7e4b04d713bd admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:54.046271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baf00b-096d-4e9c-a8ff-99ce749667e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:54.061096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baf00b-096d-4e9c-a8ff-99ce749667e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:54.061972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baf00b-096d-4e9c-a8ff-99ce749667e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:54.063120 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baf00b-096d-4e9c-a8ff-99ce749667e1 admin admin] PATH_INFO: `/v3/projects/e826d7d80f2540e8931657120df02b72` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:54.065847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48baf00b-096d-4e9c-a8ff-99ce749667e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e826d7d80f2540e8931657120df02b72)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:54.068199 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168808c-720b-4aab-a60f-7e4b04d713bd admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:54.071458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48baf00b-096d-4e9c-a8ff-99ce749667e1 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:54.085092 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fae77-9197-4bad-8e4d-26415679ae7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:54.085992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fae77-9197-4bad-8e4d-26415679ae7e admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:54.087511 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fae77-9197-4bad-8e4d-26415679ae7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:54.088083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fae77-9197-4bad-8e4d-26415679ae7e admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:54.089452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348fae77-9197-4bad-8e4d-26415679ae7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:54.094227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348fae77-9197-4bad-8e4d-26415679ae7e admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:54.304407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 889/1763] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:54 2022] POST /identity/v3/users => generated 301 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:54.306460 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fae77-9197-4bad-8e4d-26415679ae7e admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:54.326104 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fae77-9197-4bad-8e4d-26415679ae7e admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:54.356600 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:54.357319 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea26822-5eec-4b22-93fc-01103b0c53f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:54.359906 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea26822-5eec-4b22-93fc-01103b0c53f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:54.363490 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea26822-5eec-4b22-93fc-01103b0c53f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:54.374409 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea26822-5eec-4b22-93fc-01103b0c53f0 admin admin] PATH_INFO: `/v3/projects/92f6eaad43d6446e885c8aae4366010a/users/5a7ecd5c59d34ed0ab9507ad3404744e/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:54.380785 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea26822-5eec-4b22-93fc-01103b0c53f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f6eaad43d6446e885c8aae4366010a, user_id=5a7ecd5c59d34ed0ab9507ad3404744e, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:54.384407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-48baf00b-096d-4e9c-a8ff-99ce749667e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e826d7d80f2540e8931657120df02b72', 'request_id': 'req-48baf00b-096d-4e9c-a8ff-99ce749667e1'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:54.390193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea26822-5eec-4b22-93fc-01103b0c53f0 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:54.393998 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-48baf00b-096d-4e9c-a8ff-99ce749667e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e826d7d80f2540e8931657120df02b72', 'request_id': 'req-48baf00b-096d-4e9c-a8ff-99ce749667e1'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:54.405233 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-48baf00b-096d-4e9c-a8ff-99ce749667e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e826d7d80f2540e8931657120df02b72', 'request_id': 'req-48baf00b-096d-4e9c-a8ff-99ce749667e1'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:54.409183 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 890/1764] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:54 2022] PUT /identity/v3/projects/92f6eaad43d6446e885c8aae4366010a/users/5a7ecd5c59d34ed0ab9507ad3404744e/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:54.423907 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea26822-5eec-4b22-93fc-01103b0c53f0 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:54.552385 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-48baf00b-096d-4e9c-a8ff-99ce749667e1 admin admin] The token cache is being invalidate because project e826d7d80f2540e8931657120df02b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:13:54.552958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-48baf00b-096d-4e9c-a8ff-99ce749667e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:54.555655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-48baf00b-096d-4e9c-a8ff-99ce749667e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:54.561145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 875/1765] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:13:53 2022] DELETE /identity/v3/projects/e826d7d80f2540e8931657120df02b72 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:54.564100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baf00b-096d-4e9c-a8ff-99ce749667e1 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:54.571558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-48baf00b-096d-4e9c-a8ff-99ce749667e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:54.629393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea26822-5eec-4b22-93fc-01103b0c53f0 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:54.653404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492f897-1773-4fe2-bd9c-5c8801549853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:54.654776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492f897-1773-4fe2-bd9c-5c8801549853 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:54.655257 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492f897-1773-4fe2-bd9c-5c8801549853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:54.655765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492f897-1773-4fe2-bd9c-5c8801549853 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:54.657264 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492f897-1773-4fe2-bd9c-5c8801549853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:54.667565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492f897-1773-4fe2-bd9c-5c8801549853 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:54.713982 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baf00b-096d-4e9c-a8ff-99ce749667e1 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:54.723969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 891/1766] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:54.726725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b24a1-bfc3-4125-b69f-d429a687ce25 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:54.727234 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b24a1-bfc3-4125-b69f-d429a687ce25 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:54.727747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b24a1-bfc3-4125-b69f-d429a687ce25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:54.790903 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e477fe-fd94-424c-af10-54a9b206d11d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:54.792019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e477fe-fd94-424c-af10-54a9b206d11d service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:54.792390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e477fe-fd94-424c-af10-54a9b206d11d service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:54.792751 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e477fe-fd94-424c-af10-54a9b206d11d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:54.795338 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e0e477fe-fd94-424c-af10-54a9b206d11d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:54.846998 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-204b24a1-bfc3-4125-b69f-d429a687ce25 None None] MFA Rules not processed for user `5a7ecd5c59d34ed0ab9507ad3404744e`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:54.849958 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-204b24a1-bfc3-4125-b69f-d429a687ce25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:54.908527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e477fe-fd94-424c-af10-54a9b206d11d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:54.913057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e477fe-fd94-424c-af10-54a9b206d11d service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:54.958804 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 876/1767] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:54 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:54.962010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3487433-f857-406d-bc0b-2728fddf7df8 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:54.962526 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3487433-f857-406d-bc0b-2728fddf7df8 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:54.972184 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3487433-f857-406d-bc0b-2728fddf7df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:55.137882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-c3487433-f857-406d-bc0b-2728fddf7df8 None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:55.140488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c3487433-f857-406d-bc0b-2728fddf7df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:55.149974 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:55.158674 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 892/1768] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:54 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:55.160110 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b24a1-bfc3-4125-b69f-d429a687ce25 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:55.164623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-204b24a1-bfc3-4125-b69f-d429a687ce25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:55.186652 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b24a1-bfc3-4125-b69f-d429a687ce25 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:55.186652 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fe2e2-b29e-49b1-9117-fca30611ca3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:55.189229 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fe2e2-b29e-49b1-9117-fca30611ca3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:55.189229 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fe2e2-b29e-49b1-9117-fca30611ca3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:55.189229 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fe2e2-b29e-49b1-9117-fca30611ca3f admin admin] PATH_INFO: `/v3/users/86fe3352ce7a42f0b2bdac7184e643d5` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:55.189586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573fe2e2-b29e-49b1-9117-fca30611ca3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=86fe3352ce7a42f0b2bdac7184e643d5)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:55.191465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573fe2e2-b29e-49b1-9117-fca30611ca3f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:55.195293 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:55.350160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 877/1769] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Jul 18 15:13:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:55.354627 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3487433-f857-406d-bc0b-2728fddf7df8 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:55.354627 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c3487433-f857-406d-bc0b-2728fddf7df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:55.389689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3487433-f857-406d-bc0b-2728fddf7df8 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:55.402645 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615f445-e2f7-42f5-9e48-02d1b8a68a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:55.404801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615f445-e2f7-42f5-9e48-02d1b8a68a13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:55.405227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615f445-e2f7-42f5-9e48-02d1b8a68a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:55.405407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615f445-e2f7-42f5-9e48-02d1b8a68a13 admin admin] PATH_INFO: `/v3/projects/16321e1b87854fa4a67081b4c77f663d/users/4aa5c98f782b405ab3a50dcc5cbef9f1/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:55.423235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615f445-e2f7-42f5-9e48-02d1b8a68a13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16321e1b87854fa4a67081b4c77f663d, user_id=4aa5c98f782b405ab3a50dcc5cbef9f1, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:55.425995 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615f445-e2f7-42f5-9e48-02d1b8a68a13 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:55.492006 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-573fe2e2-b29e-49b1-9117-fca30611ca3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86fe3352ce7a42f0b2bdac7184e643d5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:55.501753 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-573fe2e2-b29e-49b1-9117-fca30611ca3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86fe3352ce7a42f0b2bdac7184e643d5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:55.502516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-573fe2e2-b29e-49b1-9117-fca30611ca3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86fe3352ce7a42f0b2bdac7184e643d5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:55.553144 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-573fe2e2-b29e-49b1-9117-fca30611ca3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86fe3352ce7a42f0b2bdac7184e643d5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:55.602617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 878/1770] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:55 2022] PUT /identity/v3/projects/16321e1b87854fa4a67081b4c77f663d/users/4aa5c98f782b405ab3a50dcc5cbef9f1/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:55.603730 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615f445-e2f7-42f5-9e48-02d1b8a68a13 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:55.611909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-3615f445-e2f7-42f5-9e48-02d1b8a68a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:55.739212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-573fe2e2-b29e-49b1-9117-fca30611ca3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86fe3352ce7a42f0b2bdac7184e643d5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:55.772245 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 893/1771] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:13:55 2022] DELETE /identity/v3/users/86fe3352ce7a42f0b2bdac7184e643d5 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:55.774685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fe2e2-b29e-49b1-9117-fca30611ca3f admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:55.776951 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615f445-e2f7-42f5-9e48-02d1b8a68a13 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:55.782054 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-573fe2e2-b29e-49b1-9117-fca30611ca3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:55.955494 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afffab1-bcd3-40a2-901d-cf90453f7f9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:55.956214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afffab1-bcd3-40a2-901d-cf90453f7f9b service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:55.956596 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afffab1-bcd3-40a2-901d-cf90453f7f9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:55.956915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afffab1-bcd3-40a2-901d-cf90453f7f9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:55.966300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7afffab1-bcd3-40a2-901d-cf90453f7f9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:55.985089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fe2e2-b29e-49b1-9117-fca30611ca3f admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:56.000462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b35fad-e805-4ba1-962a-d0ace98e72b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:56.001759 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b35fad-e805-4ba1-962a-d0ace98e72b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:56.002289 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b35fad-e805-4ba1-962a-d0ace98e72b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:56.002815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b35fad-e805-4ba1-962a-d0ace98e72b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:56.006365 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b35fad-e805-4ba1-962a-d0ace98e72b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:56.008737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b35fad-e805-4ba1-962a-d0ace98e72b0 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:56.048558 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 894/1772] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:13:56.053905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b35fad-e805-4ba1-962a-d0ace98e72b0 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:56.058187 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:56.062602 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-35b35fad-e805-4ba1-962a-d0ace98e72b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:56.104340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afffab1-bcd3-40a2-901d-cf90453f7f9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:56.104340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afffab1-bcd3-40a2-901d-cf90453f7f9b service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:56.181857 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 879/1773] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:55 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:56.185044 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afffab1-bcd3-40a2-901d-cf90453f7f9b service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:56.218747 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afffab1-bcd3-40a2-901d-cf90453f7f9b service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:56.231900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319f87d-3e3d-4f52-8b68-ed8f36066b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:56.233302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319f87d-3e3d-4f52-8b68-ed8f36066b58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:56.233720 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319f87d-3e3d-4f52-8b68-ed8f36066b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:56.234124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319f87d-3e3d-4f52-8b68-ed8f36066b58 admin admin] PATH_INFO: `/v3/projects/16321e1b87854fa4a67081b4c77f663d/users/4aa5c98f782b405ab3a50dcc5cbef9f1/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:56.237426 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319f87d-3e3d-4f52-8b68-ed8f36066b58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16321e1b87854fa4a67081b4c77f663d, user_id=4aa5c98f782b405ab3a50dcc5cbef9f1, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:56.242080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319f87d-3e3d-4f52-8b68-ed8f36066b58 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:56.257149 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 880/1774] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:13:56 2022] PUT /identity/v3/projects/16321e1b87854fa4a67081b4c77f663d/users/4aa5c98f782b405ab3a50dcc5cbef9f1/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:56.258912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f6ab9-5da1-4d18-8dd9-be5b9845ece2 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:56.259335 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f6ab9-5da1-4d18-8dd9-be5b9845ece2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:56.260141 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f6ab9-5da1-4d18-8dd9-be5b9845ece2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:56.294352 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:56.323836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-9a0f6ab9-5da1-4d18-8dd9-be5b9845ece2 None None] MFA Rules not processed for user `fd86979e0f0d4a4e9921ee5704901f74`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:56.327264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9a0f6ab9-5da1-4d18-8dd9-be5b9845ece2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:56.331647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b35fad-e805-4ba1-962a-d0ace98e72b0 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:56.448994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 881/1775] 10.222.0.91 () {58 vars in 1015 bytes} [Mon Jul 18 15:13:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:56.450820 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e6565-f75f-41a2-82cd-a8e1bc3a7196 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:56.451194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e6565-f75f-41a2-82cd-a8e1bc3a7196 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:56.452511 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e6565-f75f-41a2-82cd-a8e1bc3a7196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:56.543671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-509e6565-f75f-41a2-82cd-a8e1bc3a7196 None None] MFA Rules not processed for user `4aa5c98f782b405ab3a50dcc5cbef9f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:56.545809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-509e6565-f75f-41a2-82cd-a8e1bc3a7196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:56.747879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc20b33-05e6-49e7-988b-6fa6687b49f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:56.749956 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc20b33-05e6-49e7-988b-6fa6687b49f5 service swift] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:56.750344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc20b33-05e6-49e7-988b-6fa6687b49f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:56.751545 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc20b33-05e6-49e7-988b-6fa6687b49f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:56.755898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7dc20b33-05e6-49e7-988b-6fa6687b49f5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:56.829981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 882/1776] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:56.849554 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e6565-f75f-41a2-82cd-a8e1bc3a7196 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:56.851144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-509e6565-f75f-41a2-82cd-a8e1bc3a7196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:56.951821 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc20b33-05e6-49e7-988b-6fa6687b49f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:56.953605 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc20b33-05e6-49e7-988b-6fa6687b49f5 service swift] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:56.972396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e6565-f75f-41a2-82cd-a8e1bc3a7196 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:56.986114 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78308bc1-f2d7-489a-9380-19d50e6e18b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:56.986834 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78308bc1-f2d7-489a-9380-19d50e6e18b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:56.987317 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78308bc1-f2d7-489a-9380-19d50e6e18b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:56.987892 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78308bc1-f2d7-489a-9380-19d50e6e18b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:57.014947 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 895/1777] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:13:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 964 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 15:13:57.015312 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78308bc1-f2d7-489a-9380-19d50e6e18b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:57.016644 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc20b33-05e6-49e7-988b-6fa6687b49f5 service swift] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:57.017625 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78308bc1-f2d7-489a-9380-19d50e6e18b5 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:57.037516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7dc20b33-05e6-49e7-988b-6fa6687b49f5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:57.053830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 883/1778] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:57.090623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc20b33-05e6-49e7-988b-6fa6687b49f5 service swift] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:57.106018 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22f96a-e8ff-4fea-8ac3-4866c85b43c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:57.111042 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22f96a-e8ff-4fea-8ac3-4866c85b43c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:57.111738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22f96a-e8ff-4fea-8ac3-4866c85b43c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:57.112329 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22f96a-e8ff-4fea-8ac3-4866c85b43c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:57.147632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22f96a-e8ff-4fea-8ac3-4866c85b43c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:57.149621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22f96a-e8ff-4fea-8ac3-4866c85b43c8 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:57.196491 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 896/1779] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:57.402005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78308bc1-f2d7-489a-9380-19d50e6e18b5 service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:57.403874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-78308bc1-f2d7-489a-9380-19d50e6e18b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:57.435877 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:57.706096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78308bc1-f2d7-489a-9380-19d50e6e18b5 service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:57.774043 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b1b66-833e-4b44-9bb0-d7f34f5b2026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:57.781092 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b1b66-833e-4b44-9bb0-d7f34f5b2026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:57.781969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b1b66-833e-4b44-9bb0-d7f34f5b2026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:57.791392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b1b66-833e-4b44-9bb0-d7f34f5b2026 admin admin] PATH_INFO: `/v3/projects/591112c245214d9eb9cde5df54feebdb` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:57.813339 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b1b66-833e-4b44-9bb0-d7f34f5b2026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=591112c245214d9eb9cde5df54feebdb)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:57.818888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b1b66-833e-4b44-9bb0-d7f34f5b2026 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:57.887637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22f96a-e8ff-4fea-8ac3-4866c85b43c8 service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:57.918418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22f96a-e8ff-4fea-8ac3-4866c85b43c8 service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:57.945483 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97332b93-7b1c-472e-97a9-7d346a8f8124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:57.946392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97332b93-7b1c-472e-97a9-7d346a8f8124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:57.946715 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97332b93-7b1c-472e-97a9-7d346a8f8124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:57.947000 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97332b93-7b1c-472e-97a9-7d346a8f8124 admin admin] PATH_INFO: `/v3/users/58a41d81210b4e408433c92957c010df` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:57.961826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97332b93-7b1c-472e-97a9-7d346a8f8124 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58a41d81210b4e408433c92957c010df)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:57.969264 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97332b93-7b1c-472e-97a9-7d346a8f8124 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:58.200217 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-538b1b66-833e-4b44-9bb0-d7f34f5b2026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '591112c245214d9eb9cde5df54feebdb', 'request_id': 'req-538b1b66-833e-4b44-9bb0-d7f34f5b2026'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:58.233493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-538b1b66-833e-4b44-9bb0-d7f34f5b2026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '591112c245214d9eb9cde5df54feebdb', 'request_id': 'req-538b1b66-833e-4b44-9bb0-d7f34f5b2026'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:58.256197 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-538b1b66-833e-4b44-9bb0-d7f34f5b2026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '591112c245214d9eb9cde5df54feebdb', 'request_id': 'req-538b1b66-833e-4b44-9bb0-d7f34f5b2026'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:58.493653 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-538b1b66-833e-4b44-9bb0-d7f34f5b2026 admin admin] The token cache is being invalidate because project 591112c245214d9eb9cde5df54feebdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:13:58.494206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-538b1b66-833e-4b44-9bb0-d7f34f5b2026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:58.500168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-538b1b66-833e-4b44-9bb0-d7f34f5b2026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:58.503096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-97332b93-7b1c-472e-97a9-7d346a8f8124 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58a41d81210b4e408433c92957c010df'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:58.519420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 884/1780] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:13:57 2022] DELETE /identity/v3/projects/591112c245214d9eb9cde5df54feebdb => generated 0 bytes in 1117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:58.521101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b1b66-833e-4b44-9bb0-d7f34f5b2026 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:58.524358 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-538b1b66-833e-4b44-9bb0-d7f34f5b2026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:58.529528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-97332b93-7b1c-472e-97a9-7d346a8f8124 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58a41d81210b4e408433c92957c010df'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:58.531499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-97332b93-7b1c-472e-97a9-7d346a8f8124 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58a41d81210b4e408433c92957c010df'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:58.585006 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-97332b93-7b1c-472e-97a9-7d346a8f8124 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58a41d81210b4e408433c92957c010df'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:58.711548 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-97332b93-7b1c-472e-97a9-7d346a8f8124 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58a41d81210b4e408433c92957c010df'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:13:58.714969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:58.726024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 897/1781] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:13:57 2022] DELETE /identity/v3/users/58a41d81210b4e408433c92957c010df => generated 0 bytes in 835 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:13:58.726024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f45ad-bd63-4bf1-af22-dc8b06d2e0da None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:58.726024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f45ad-bd63-4bf1-af22-dc8b06d2e0da None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:58.726024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f45ad-bd63-4bf1-af22-dc8b06d2e0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:58.775981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-406f45ad-bd63-4bf1-af22-dc8b06d2e0da None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:58.779291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-406f45ad-bd63-4bf1-af22-dc8b06d2e0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:58.788045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b1b66-833e-4b44-9bb0-d7f34f5b2026 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:59.029871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 898/1782] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:59.032802 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606418b-7355-4e54-a7db-c1e55e9420cd None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:59.033295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606418b-7355-4e54-a7db-c1e55e9420cd None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:59.033776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606418b-7355-4e54-a7db-c1e55e9420cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:59.066339 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:13:59.118381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2f068-39f9-4fb4-9b16-bd97194667be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:59.119326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2f068-39f9-4fb4-9b16-bd97194667be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:59.119615 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2f068-39f9-4fb4-9b16-bd97194667be admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:59.120069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2f068-39f9-4fb4-9b16-bd97194667be admin admin] PATH_INFO: `/v3/users/dbf97c999c66499bbbc4252159f406b5` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:59.125325 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-7606418b-7355-4e54-a7db-c1e55e9420cd None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:59.127103 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2f068-39f9-4fb4-9b16-bd97194667be admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbf97c999c66499bbbc4252159f406b5)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:59.133912 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7606418b-7355-4e54-a7db-c1e55e9420cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:59.136874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2f068-39f9-4fb4-9b16-bd97194667be admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:59.165768 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 899/1783] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:59.286654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b63e68-f6ce-4518-9257-350f9062c9b8 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:59.286654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b63e68-f6ce-4518-9257-350f9062c9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:59.286654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b63e68-f6ce-4518-9257-350f9062c9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:59.286654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-83b63e68-f6ce-4518-9257-350f9062c9b8 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:13:59.286654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-83b63e68-f6ce-4518-9257-350f9062c9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:59.286654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 900/1784] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:13:59.288459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b63e68-f6ce-4518-9257-350f9062c9b8 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:59.288459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b63e68-f6ce-4518-9257-350f9062c9b8 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:59.288459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1a3f5-ef80-40a8-9685-303effcd7ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:59.288688 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1a3f5-ef80-40a8-9685-303effcd7ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:59.289493 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1a3f5-ef80-40a8-9685-303effcd7ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:59.290123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1a3f5-ef80-40a8-9685-303effcd7ab9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:59.292141 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1a3f5-ef80-40a8-9685-303effcd7ab9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:13:59.292712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1a3f5-ef80-40a8-9685-303effcd7ab9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:59.296440 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1a3f5-ef80-40a8-9685-303effcd7ab9 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:59.343676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 901/1785] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:13:59.343676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1a3f5-ef80-40a8-9685-303effcd7ab9 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:59.343676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-bff1a3f5-ef80-40a8-9685-303effcd7ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:13:59.540266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1a3f5-ef80-40a8-9685-303effcd7ab9 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:59.549831 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d05d0-ab4d-47c8-a857-810e78157174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:59.550649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d05d0-ab4d-47c8-a857-810e78157174 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:59.550965 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d05d0-ab4d-47c8-a857-810e78157174 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:59.551275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d05d0-ab4d-47c8-a857-810e78157174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:59.587684 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d05d0-ab4d-47c8-a857-810e78157174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:59.597844 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d05d0-ab4d-47c8-a857-810e78157174 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:59.634615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 902/1786] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:13:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:13:59.636999 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d05d0-ab4d-47c8-a857-810e78157174 service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:13:59.656344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d05d0-ab4d-47c8-a857-810e78157174 service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:13:59.669474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c49a51-3100-4572-baba-92ffaa4012ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:13:59.670798 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c49a51-3100-4572-baba-92ffaa4012ee admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:13:59.671320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c49a51-3100-4572-baba-92ffaa4012ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:13:59.671784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c49a51-3100-4572-baba-92ffaa4012ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:13:59.673105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c49a51-3100-4572-baba-92ffaa4012ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:13:59.676569 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c49a51-3100-4572-baba-92ffaa4012ee admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:13:59.977630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 903/1787] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:13:59 2022] POST /identity/v3/projects => generated 373 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:13:59.982426 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c49a51-3100-4572-baba-92ffaa4012ee admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:00.041041 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-51a2f068-39f9-4fb4-9b16-bd97194667be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbf97c999c66499bbbc4252159f406b5'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:00.045768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-51a2f068-39f9-4fb4-9b16-bd97194667be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbf97c999c66499bbbc4252159f406b5'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:00.046914 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-51a2f068-39f9-4fb4-9b16-bd97194667be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbf97c999c66499bbbc4252159f406b5'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:00.147303 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c49a51-3100-4572-baba-92ffaa4012ee admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:00.155155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039811be-4499-40e3-99a8-fa42936b5dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:00.156751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039811be-4499-40e3-99a8-fa42936b5dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:00.157599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039811be-4499-40e3-99a8-fa42936b5dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:00.157599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039811be-4499-40e3-99a8-fa42936b5dd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:00.158331 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039811be-4499-40e3-99a8-fa42936b5dd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:00.163488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039811be-4499-40e3-99a8-fa42936b5dd6 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:00.175349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-51a2f068-39f9-4fb4-9b16-bd97194667be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbf97c999c66499bbbc4252159f406b5'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:00.201511 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-51a2f068-39f9-4fb4-9b16-bd97194667be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbf97c999c66499bbbc4252159f406b5'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:00.245761 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 885/1788] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:13:58 2022] DELETE /identity/v3/users/dbf97c999c66499bbbc4252159f406b5 => generated 0 bytes in 1726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:00.256036 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10e5e0-61da-4ed4-b15e-ab06e228f57a None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:00.257186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10e5e0-61da-4ed4-b15e-ab06e228f57a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:00.257186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10e5e0-61da-4ed4-b15e-ab06e228f57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:00.301009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-bb10e5e0-61da-4ed4-b15e-ab06e228f57a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:00.302908 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-bb10e5e0-61da-4ed4-b15e-ab06e228f57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:00.304421 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 904/1789] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:13:59 2022] POST /identity/v3/users => generated 287 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:00.323962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039811be-4499-40e3-99a8-fa42936b5dd6 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:00.337560 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-039811be-4499-40e3-99a8-fa42936b5dd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:00.375373 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:00.423305 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039811be-4499-40e3-99a8-fa42936b5dd6 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:00.436024 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 886/1790] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:00.443781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4fa04-3df2-4004-ab61-b2ab37a671fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:00.451704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4fa04-3df2-4004-ab61-b2ab37a671fc admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:00.451914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4fa04-3df2-4004-ab61-b2ab37a671fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:00.452562 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4fa04-3df2-4004-ab61-b2ab37a671fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:00.453376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4fa04-3df2-4004-ab61-b2ab37a671fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:00.455330 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4fa04-3df2-4004-ab61-b2ab37a671fc admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:00.482888 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 905/1791] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:00.487008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10e5e0-61da-4ed4-b15e-ab06e228f57a None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:00.508037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10e5e0-61da-4ed4-b15e-ab06e228f57a None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:00.531542 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d86c51-9bf4-4333-a685-92e417e395b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:00.535074 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d86c51-9bf4-4333-a685-92e417e395b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:00.535442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d86c51-9bf4-4333-a685-92e417e395b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:00.535681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d86c51-9bf4-4333-a685-92e417e395b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:00.539696 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d86c51-9bf4-4333-a685-92e417e395b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:00.544579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d86c51-9bf4-4333-a685-92e417e395b5 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:00.577898 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 887/1792] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:00.578640 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4fa04-3df2-4004-ab61-b2ab37a671fc admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:00.602170 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d86c51-9bf4-4333-a685-92e417e395b5 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:00.617176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-48d86c51-9bf4-4333-a685-92e417e395b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:00.667885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4fa04-3df2-4004-ab61-b2ab37a671fc admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:00.693123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dcc854-deac-4023-aa7b-2a732994f3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:00.695821 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcc854-deac-4023-aa7b-2a732994f3bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:00.697205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcc854-deac-4023-aa7b-2a732994f3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:00.697922 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcc854-deac-4023-aa7b-2a732994f3bd admin admin] PATH_INFO: `/v3/projects/130badbb699444e6b865087023dddf29/users/875e46202d454079ae9d44bdacc8d077/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:00.750117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dcc854-deac-4023-aa7b-2a732994f3bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=130badbb699444e6b865087023dddf29, user_id=875e46202d454079ae9d44bdacc8d077, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:00.755667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dcc854-deac-4023-aa7b-2a732994f3bd admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:00.842413 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d86c51-9bf4-4333-a685-92e417e395b5 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:00.875048 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 906/1793] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:00 2022] PUT /identity/v3/projects/130badbb699444e6b865087023dddf29/users/875e46202d454079ae9d44bdacc8d077/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:00.879590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f17a1-5d67-4e4d-b24a-bc08f6db05d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:00.880452 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f17a1-5d67-4e4d-b24a-bc08f6db05d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:00.880871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f17a1-5d67-4e4d-b24a-bc08f6db05d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:00.881441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f17a1-5d67-4e4d-b24a-bc08f6db05d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:00.885434 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fedf0-cdb2-4d11-89cb-c7949ec0d54d None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:00.885434 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fedf0-cdb2-4d11-89cb-c7949ec0d54d None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:00.885728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fedf0-cdb2-4d11-89cb-c7949ec0d54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:00.919353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f17a1-5d67-4e4d-b24a-bc08f6db05d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:00.930394 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f17a1-5d67-4e4d-b24a-bc08f6db05d0 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:00.940144 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-fc0fedf0-cdb2-4d11-89cb-c7949ec0d54d None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:00.941738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-fc0fedf0-cdb2-4d11-89cb-c7949ec0d54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:00.977210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 888/1794] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:01.048718 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 907/1795] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:01.183790 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f17a1-5d67-4e4d-b24a-bc08f6db05d0 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:01.185703 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-bf8f17a1-5d67-4e4d-b24a-bc08f6db05d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:01.203631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:01.214961 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f17a1-5d67-4e4d-b24a-bc08f6db05d0 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:01.316894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3628721-124a-416f-8a2e-fa4884fe18c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:01.319400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3628721-124a-416f-8a2e-fa4884fe18c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:01.319954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3628721-124a-416f-8a2e-fa4884fe18c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:01.320406 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3628721-124a-416f-8a2e-fa4884fe18c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:01.331837 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3628721-124a-416f-8a2e-fa4884fe18c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:01.338914 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3628721-124a-416f-8a2e-fa4884fe18c2 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:01.381521 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 889/1796] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:02.840349 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a0d33-860b-4302-bb6e-217fefa5ad28 None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:02.844835 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a0d33-860b-4302-bb6e-217fefa5ad28 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:02.844835 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a0d33-860b-4302-bb6e-217fefa5ad28 None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:02.850097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 908/1797] 10.222.0.83 () {54 vars in 921 bytes} [Mon Jul 18 15:14:02 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:14:02.932791 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3628721-124a-416f-8a2e-fa4884fe18c2 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:02.938061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a3628721-124a-416f-8a2e-fa4884fe18c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:02.957182 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3628721-124a-416f-8a2e-fa4884fe18c2 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:02.975007 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:02.976184 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:02.976921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:02.977558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4 admin admin] PATH_INFO: `/v3/projects/8b708ec6dc5e41f4b0b16583391049ff` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:03.002990 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b708ec6dc5e41f4b0b16583391049ff)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:03.006566 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:03.035719 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862a365-a5f3-41b0-a4c9-387679e677fa None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:03.036780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862a365-a5f3-41b0-a4c9-387679e677fa None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:03.037436 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862a365-a5f3-41b0-a4c9-387679e677fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:03.096948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-7862a365-a5f3-41b0-a4c9-387679e677fa None None] MFA Rules not processed for user `7e1355d18fb049468651b3675c217d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:03.105858 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7862a365-a5f3-41b0-a4c9-387679e677fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:03.265100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b708ec6dc5e41f4b0b16583391049ff', 'request_id': 'req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:03.280011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b708ec6dc5e41f4b0b16583391049ff', 'request_id': 'req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:03.290683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b708ec6dc5e41f4b0b16583391049ff', 'request_id': 'req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:03.333717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 909/1798] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Jul 18 15:14:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:03.380991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4 admin admin] The token cache is being invalidate because project 8b708ec6dc5e41f4b0b16583391049ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:03.381617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:03.383530 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:03.390305 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 890/1799] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:02 2022] DELETE /identity/v3/projects/8b708ec6dc5e41f4b0b16583391049ff => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:03.399645 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862a365-a5f3-41b0-a4c9-387679e677fa None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:03.420267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862a365-a5f3-41b0-a4c9-387679e677fa None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:03.587911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:03.664136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7f856-015a-4ab5-ae41-e25bc74b2466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:03.665711 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7f856-015a-4ab5-ae41-e25bc74b2466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:03.669288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7f856-015a-4ab5-ae41-e25bc74b2466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:03.671723 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7f856-015a-4ab5-ae41-e25bc74b2466 admin admin] PATH_INFO: `/v3/users/4aa5c98f782b405ab3a50dcc5cbef9f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:03.679578 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7f856-015a-4ab5-ae41-e25bc74b2466 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aa5c98f782b405ab3a50dcc5cbef9f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:03.682164 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7f856-015a-4ab5-ae41-e25bc74b2466 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:04.112256 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-60a7f856-015a-4ab5-ae41-e25bc74b2466 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aa5c98f782b405ab3a50dcc5cbef9f1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:04.162633 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-60a7f856-015a-4ab5-ae41-e25bc74b2466 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aa5c98f782b405ab3a50dcc5cbef9f1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:04.163659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-60a7f856-015a-4ab5-ae41-e25bc74b2466 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aa5c98f782b405ab3a50dcc5cbef9f1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:04.220451 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:04.222797 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:04.238291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-60a7f856-015a-4ab5-ae41-e25bc74b2466 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aa5c98f782b405ab3a50dcc5cbef9f1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:04.306332 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-60a7f856-015a-4ab5-ae41-e25bc74b2466 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aa5c98f782b405ab3a50dcc5cbef9f1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:04.315070 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 910/1800] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:03 2022] DELETE /identity/v3/users/4aa5c98f782b405ab3a50dcc5cbef9f1 => generated 0 bytes in 920 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:04.319800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7f856-015a-4ab5-ae41-e25bc74b2466 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:04.320905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-60a7f856-015a-4ab5-ae41-e25bc74b2466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:04.475705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5446a-ad72-44a0-81c2-aca9a0a95ba4 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:04.562361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:04.569379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7f856-015a-4ab5-ae41-e25bc74b2466 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:04.584929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f8eec-97f7-4d81-a4f7-db43dc58bb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:04.586562 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f8eec-97f7-4d81-a4f7-db43dc58bb08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:04.586963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f8eec-97f7-4d81-a4f7-db43dc58bb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:04.587375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f8eec-97f7-4d81-a4f7-db43dc58bb08 admin admin] PATH_INFO: `/v3/projects/19ae2153d19949f5a9eab06296eaa333` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:04.590851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f8eec-97f7-4d81-a4f7-db43dc58bb08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19ae2153d19949f5a9eab06296eaa333)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:04.599440 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f8eec-97f7-4d81-a4f7-db43dc58bb08 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:04.663406 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb948765-6473-4f6a-b54f-b207bcbcf54e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:04.665783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb948765-6473-4f6a-b54f-b207bcbcf54e service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:04.666181 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb948765-6473-4f6a-b54f-b207bcbcf54e service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:04.666588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb948765-6473-4f6a-b54f-b207bcbcf54e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:04.672184 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-bb948765-6473-4f6a-b54f-b207bcbcf54e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:04.701713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb948765-6473-4f6a-b54f-b207bcbcf54e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:04.703772 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb948765-6473-4f6a-b54f-b207bcbcf54e service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:04.751430 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 891/1801] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:04.806693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-263f8eec-97f7-4d81-a4f7-db43dc58bb08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19ae2153d19949f5a9eab06296eaa333', 'request_id': 'req-263f8eec-97f7-4d81-a4f7-db43dc58bb08'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:04.816368 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-263f8eec-97f7-4d81-a4f7-db43dc58bb08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19ae2153d19949f5a9eab06296eaa333', 'request_id': 'req-263f8eec-97f7-4d81-a4f7-db43dc58bb08'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:04.905684 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-263f8eec-97f7-4d81-a4f7-db43dc58bb08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19ae2153d19949f5a9eab06296eaa333', 'request_id': 'req-263f8eec-97f7-4d81-a4f7-db43dc58bb08'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:04.943515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-263f8eec-97f7-4d81-a4f7-db43dc58bb08 admin admin] The token cache is being invalidate because project 19ae2153d19949f5a9eab06296eaa333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:04.943827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-263f8eec-97f7-4d81-a4f7-db43dc58bb08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:04.946172 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-263f8eec-97f7-4d81-a4f7-db43dc58bb08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:04.973685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 911/1802] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:04 2022] DELETE /identity/v3/projects/19ae2153d19949f5a9eab06296eaa333 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:04.973805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb948765-6473-4f6a-b54f-b207bcbcf54e service nova] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:04.990194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb948765-6473-4f6a-b54f-b207bcbcf54e service nova] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:05.028772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:05.139743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d111803-8937-41f9-98e7-cc58d7c27c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:05.141143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d111803-8937-41f9-98e7-cc58d7c27c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:05.141711 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d111803-8937-41f9-98e7-cc58d7c27c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:05.143241 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d111803-8937-41f9-98e7-cc58d7c27c55 admin admin] PATH_INFO: `/v3/users/5a7ecd5c59d34ed0ab9507ad3404744e` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:05.150015 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d111803-8937-41f9-98e7-cc58d7c27c55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a7ecd5c59d34ed0ab9507ad3404744e)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:05.190901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d111803-8937-41f9-98e7-cc58d7c27c55 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:05.567775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0d111803-8937-41f9-98e7-cc58d7c27c55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a7ecd5c59d34ed0ab9507ad3404744e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:05.572637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0d111803-8937-41f9-98e7-cc58d7c27c55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a7ecd5c59d34ed0ab9507ad3404744e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:05.576570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0d111803-8937-41f9-98e7-cc58d7c27c55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a7ecd5c59d34ed0ab9507ad3404744e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:05.669376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85b52f-b0aa-4423-a8d2-e85067a991dc None None] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:05.669983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85b52f-b0aa-4423-a8d2-e85067a991dc None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:05.669983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85b52f-b0aa-4423-a8d2-e85067a991dc None None] PATH_INFO: `/` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:05.671682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 912/1803] 10.222.0.83 () {54 vars in 921 bytes} [Mon Jul 18 15:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:14:05.712145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0d111803-8937-41f9-98e7-cc58d7c27c55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a7ecd5c59d34ed0ab9507ad3404744e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:05.731217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faed34-007e-45f2-8c7e-3b91fa9b58c4 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:05.732165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faed34-007e-45f2-8c7e-3b91fa9b58c4 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:05.732165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faed34-007e-45f2-8c7e-3b91fa9b58c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:05.765857 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0d111803-8937-41f9-98e7-cc58d7c27c55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a7ecd5c59d34ed0ab9507ad3404744e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:05.796097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-f5faed34-007e-45f2-8c7e-3b91fa9b58c4 None None] MFA Rules not processed for user `7e1355d18fb049468651b3675c217d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:05.796097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f5faed34-007e-45f2-8c7e-3b91fa9b58c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:05.834225 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 892/1804] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:04 2022] DELETE /identity/v3/users/5a7ecd5c59d34ed0ab9507ad3404744e => generated 0 bytes in 872 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:14:06.016832 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 913/1805] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Jul 18 15:14:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:06.625853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d111803-8937-41f9-98e7-cc58d7c27c55 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:06.629045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0d111803-8937-41f9-98e7-cc58d7c27c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:06.842711 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d111803-8937-41f9-98e7-cc58d7c27c55 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:06.877786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728c496-0add-4a8a-ab44-3e69cab5ea5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:06.878717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728c496-0add-4a8a-ab44-3e69cab5ea5a service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:06.879009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728c496-0add-4a8a-ab44-3e69cab5ea5a service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:06.881534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728c496-0add-4a8a-ab44-3e69cab5ea5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:06.908624 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728c496-0add-4a8a-ab44-3e69cab5ea5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:06.918974 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728c496-0add-4a8a-ab44-3e69cab5ea5a service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:06.965109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 893/1806] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:07.387000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:07.925682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5faed34-007e-45f2-8c7e-3b91fa9b58c4 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:07.927997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f5faed34-007e-45f2-8c7e-3b91fa9b58c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:07.989659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5faed34-007e-45f2-8c7e-3b91fa9b58c4 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:08.131575 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee30a9-eb26-4d3c-a59a-ee0dcd5fbd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:08.143016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee30a9-eb26-4d3c-a59a-ee0dcd5fbd8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:08.143016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee30a9-eb26-4d3c-a59a-ee0dcd5fbd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:08.143016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee30a9-eb26-4d3c-a59a-ee0dcd5fbd8d admin admin] PATH_INFO: `/v3/projects/16321e1b87854fa4a67081b4c77f663d` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:08.143016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee30a9-eb26-4d3c-a59a-ee0dcd5fbd8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=16321e1b87854fa4a67081b4c77f663d)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:08.167942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee30a9-eb26-4d3c-a59a-ee0dcd5fbd8d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:08.607301 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f3ee30a9-eb26-4d3c-a59a-ee0dcd5fbd8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16321e1b87854fa4a67081b4c77f663d', 'request_id': 'req-f3ee30a9-eb26-4d3c-a59a-ee0dcd5fbd8d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:08.644515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f3ee30a9-eb26-4d3c-a59a-ee0dcd5fbd8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16321e1b87854fa4a67081b4c77f663d', 'request_id': 'req-f3ee30a9-eb26-4d3c-a59a-ee0dcd5fbd8d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:08.746429 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f3ee30a9-eb26-4d3c-a59a-ee0dcd5fbd8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16321e1b87854fa4a67081b4c77f663d', 'request_id': 'req-f3ee30a9-eb26-4d3c-a59a-ee0dcd5fbd8d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:08.817752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f3ee30a9-eb26-4d3c-a59a-ee0dcd5fbd8d admin admin] The token cache is being invalidate because project 16321e1b87854fa4a67081b4c77f663d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:08.818419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f3ee30a9-eb26-4d3c-a59a-ee0dcd5fbd8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:08.835488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f3ee30a9-eb26-4d3c-a59a-ee0dcd5fbd8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:08.846642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 914/1807] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:07 2022] DELETE /identity/v3/projects/16321e1b87854fa4a67081b4c77f663d => generated 0 bytes in 922 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:08.865508 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644f5d0-3b5d-4668-b795-15ae5f8c4694 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:08.867082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644f5d0-3b5d-4668-b795-15ae5f8c4694 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:08.868461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644f5d0-3b5d-4668-b795-15ae5f8c4694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:08.947436 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-2644f5d0-3b5d-4668-b795-15ae5f8c4694 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:08.971830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2644f5d0-3b5d-4668-b795-15ae5f8c4694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:09.116307 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6455f4b-a5fa-4464-9075-a663e7a4a213 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:09.121664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6455f4b-a5fa-4464-9075-a663e7a4a213 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:09.122474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6455f4b-a5fa-4464-9075-a663e7a4a213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:09.173064 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 894/1808] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:09.191376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1b808-1fa9-47db-ad07-ab2be045524c None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:09.191799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1b808-1fa9-47db-ad07-ab2be045524c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:09.192096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1b808-1fa9-47db-ad07-ab2be045524c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:09.249339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:09.267754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-b6455f4b-a5fa-4464-9075-a663e7a4a213 None None] MFA Rules not processed for user `875e46202d454079ae9d44bdacc8d077`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:09.269037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b6455f4b-a5fa-4464-9075-a663e7a4a213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:09.277587 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-dff1b808-1fa9-47db-ad07-ab2be045524c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:09.278928 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-dff1b808-1fa9-47db-ad07-ab2be045524c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:09.313395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 895/1809] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:09.314380 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1b808-1fa9-47db-ad07-ab2be045524c None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:09.339846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1b808-1fa9-47db-ad07-ab2be045524c None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:09.356952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542921a0-720f-4fc4-8e1c-66f4ae7f090d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:09.357502 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542921a0-720f-4fc4-8e1c-66f4ae7f090d admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:09.357805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542921a0-720f-4fc4-8e1c-66f4ae7f090d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:09.358171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542921a0-720f-4fc4-8e1c-66f4ae7f090d admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:09.359342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542921a0-720f-4fc4-8e1c-66f4ae7f090d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:14:09.359674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542921a0-720f-4fc4-8e1c-66f4ae7f090d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:09.369193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542921a0-720f-4fc4-8e1c-66f4ae7f090d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:09.402928 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 896/1810] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:14:09.409235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542921a0-720f-4fc4-8e1c-66f4ae7f090d admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:09.439995 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542921a0-720f-4fc4-8e1c-66f4ae7f090d admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:09.454844 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f792baa-4425-456f-9039-6dd52625dab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:09.467224 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f792baa-4425-456f-9039-6dd52625dab5 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:09.467224 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f792baa-4425-456f-9039-6dd52625dab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:09.467224 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f792baa-4425-456f-9039-6dd52625dab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:09.467224 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f792baa-4425-456f-9039-6dd52625dab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:09.467224 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f792baa-4425-456f-9039-6dd52625dab5 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:09.486601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 915/1811] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:09.492415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6455f4b-a5fa-4464-9075-a663e7a4a213 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:09.494283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b6455f4b-a5fa-4464-9075-a663e7a4a213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:09.539349 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6455f4b-a5fa-4464-9075-a663e7a4a213 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:09.556558 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bb52d-0848-47ef-a933-be2eeb1f0d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:09.557267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bb52d-0848-47ef-a933-be2eeb1f0d6f admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:09.557564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bb52d-0848-47ef-a933-be2eeb1f0d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:09.557844 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bb52d-0848-47ef-a933-be2eeb1f0d6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:09.558583 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bb52d-0848-47ef-a933-be2eeb1f0d6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:09.560891 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bb52d-0848-47ef-a933-be2eeb1f0d6f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:09.654954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 897/1812] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:09 2022] POST /identity/v3/projects => generated 379 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:09.659477 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f792baa-4425-456f-9039-6dd52625dab5 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:09.775083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 916/1813] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:09 2022] POST /identity/v3/projects => generated 373 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:14:09.776877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bb52d-0848-47ef-a933-be2eeb1f0d6f admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:09.828041 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f792baa-4425-456f-9039-6dd52625dab5 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:09.986570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bb52d-0848-47ef-a933-be2eeb1f0d6f admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:10.000621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1768ecc-db48-4fc3-894d-2534c28dea52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:10.010672 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1768ecc-db48-4fc3-894d-2534c28dea52 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:10.010939 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1768ecc-db48-4fc3-894d-2534c28dea52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:10.011676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1768ecc-db48-4fc3-894d-2534c28dea52 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:10.012513 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1768ecc-db48-4fc3-894d-2534c28dea52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:10.019129 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1768ecc-db48-4fc3-894d-2534c28dea52 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:10.073262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43aab2-92e9-4415-a691-7dade5953803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:10.077767 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43aab2-92e9-4415-a691-7dade5953803 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:10.077767 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43aab2-92e9-4415-a691-7dade5953803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:10.077767 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43aab2-92e9-4415-a691-7dade5953803 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:10.077767 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43aab2-92e9-4415-a691-7dade5953803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:10.080110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43aab2-92e9-4415-a691-7dade5953803 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:10.200279 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 917/1814] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:09 2022] POST /identity/v3/users => generated 293 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:10.204754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1768ecc-db48-4fc3-894d-2534c28dea52 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:10.209606 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c1768ecc-db48-4fc3-894d-2534c28dea52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:10.251290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1768ecc-db48-4fc3-894d-2534c28dea52 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:10.278623 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 898/1815] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:09 2022] POST /identity/v3/users => generated 290 bytes in 619 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:14:10.278944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:10.279640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43aab2-92e9-4415-a691-7dade5953803 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:10.285570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:10.289991 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:10.292703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc admin admin] PATH_INFO: `/v3/projects/92f6eaad43d6446e885c8aae4366010a` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:10.302798 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=92f6eaad43d6446e885c8aae4366010a)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:10.306192 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:10.309241 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43aab2-92e9-4415-a691-7dade5953803 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:10.320522 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1f6de-29ed-4533-8d11-b95c47e8eb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:10.321751 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1f6de-29ed-4533-8d11-b95c47e8eb98 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:10.322447 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1f6de-29ed-4533-8d11-b95c47e8eb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:10.323031 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1f6de-29ed-4533-8d11-b95c47e8eb98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:10.324239 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1f6de-29ed-4533-8d11-b95c47e8eb98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:10.327335 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1f6de-29ed-4533-8d11-b95c47e8eb98 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:10.365579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 899/1816] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:10.366797 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1f6de-29ed-4533-8d11-b95c47e8eb98 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:10.393691 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1f6de-29ed-4533-8d11-b95c47e8eb98 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:10.423613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb80bb0-1dfd-4176-a4a7-3de25d2a5b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:10.424704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb80bb0-1dfd-4176-a4a7-3de25d2a5b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:10.425164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb80bb0-1dfd-4176-a4a7-3de25d2a5b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:10.425836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb80bb0-1dfd-4176-a4a7-3de25d2a5b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:10.433389 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb80bb0-1dfd-4176-a4a7-3de25d2a5b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:10.433389 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb80bb0-1dfd-4176-a4a7-3de25d2a5b96 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:10.469860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 900/1817] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:10.469860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb80bb0-1dfd-4176-a4a7-3de25d2a5b96 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:10.494764 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb80bb0-1dfd-4176-a4a7-3de25d2a5b96 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:10.496394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92f6eaad43d6446e885c8aae4366010a', 'request_id': 'req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:10.507032 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92f6eaad43d6446e885c8aae4366010a', 'request_id': 'req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:10.597929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92f6eaad43d6446e885c8aae4366010a', 'request_id': 'req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:10.657524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc admin admin] The token cache is being invalidate because project 92f6eaad43d6446e885c8aae4366010a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:10.658445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:10.663431 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:10.669795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b4c5f-4776-4350-97b8-3a2ec40a7a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:10.679452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 918/1818] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:10 2022] DELETE /identity/v3/projects/92f6eaad43d6446e885c8aae4366010a => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:10.681501 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:10.682185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b4c5f-4776-4350-97b8-3a2ec40a7a65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:10.692557 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:10.701115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b4c5f-4776-4350-97b8-3a2ec40a7a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:10.701832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b4c5f-4776-4350-97b8-3a2ec40a7a65 admin admin] PATH_INFO: `/v3/projects/55d46b9d2ee54c13afc48f883c648d16/users/a90ce57806964779816f3a0b542f12ef/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:10.730233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9dde22-f46d-4a08-886d-b4ea89af4cbc admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:10.743236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f9f8a-c3fe-436c-81f6-4d21f46b3b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:10.744279 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f9f8a-c3fe-436c-81f6-4d21f46b3b15 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:10.745273 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f9f8a-c3fe-436c-81f6-4d21f46b3b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:10.746135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f9f8a-c3fe-436c-81f6-4d21f46b3b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:10.753177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f9f8a-c3fe-436c-81f6-4d21f46b3b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:10.756102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b4c5f-4776-4350-97b8-3a2ec40a7a65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55d46b9d2ee54c13afc48f883c648d16, user_id=a90ce57806964779816f3a0b542f12ef, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:10.757474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f9f8a-c3fe-436c-81f6-4d21f46b3b15 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:10.759128 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b4c5f-4776-4350-97b8-3a2ec40a7a65 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:10.790107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 919/1819] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:10.791874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20378a9-ee8f-4be0-ae89-f7a3f5382d89 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:10.792205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20378a9-ee8f-4be0-ae89-f7a3f5382d89 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:10.792463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20378a9-ee8f-4be0-ae89-f7a3f5382d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:10.825913 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-d20378a9-ee8f-4be0-ae89-f7a3f5382d89 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:10.827569 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d20378a9-ee8f-4be0-ae89-f7a3f5382d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:10.851748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 920/1820] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:10.851748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20378a9-ee8f-4be0-ae89-f7a3f5382d89 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:10.860124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20378a9-ee8f-4be0-ae89-f7a3f5382d89 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:10.869845 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96711c-a288-482d-a511-20f0b9251c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:10.871697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96711c-a288-482d-a511-20f0b9251c82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:10.871951 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96711c-a288-482d-a511-20f0b9251c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:10.872461 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96711c-a288-482d-a511-20f0b9251c82 admin admin] PATH_INFO: `/v3/projects/79cba51a247e4975a8673bd975625db3/users/519eefa2426f4667a401d71c43001483/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:10.902076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96711c-a288-482d-a511-20f0b9251c82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79cba51a247e4975a8673bd975625db3, user_id=519eefa2426f4667a401d71c43001483, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:10.905141 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96711c-a288-482d-a511-20f0b9251c82 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:10.906204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 901/1821] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:10 2022] PUT /identity/v3/projects/55d46b9d2ee54c13afc48f883c648d16/users/a90ce57806964779816f3a0b542f12ef/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:10.910458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a6a87-acc6-4f3c-be05-4ac0031c45d2 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:10.912015 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a6a87-acc6-4f3c-be05-4ac0031c45d2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:10.912468 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a6a87-acc6-4f3c-be05-4ac0031c45d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:10.948716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-370a6a87-acc6-4f3c-be05-4ac0031c45d2 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:10.954817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-370a6a87-acc6-4f3c-be05-4ac0031c45d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:11.026034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 921/1822] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:10 2022] PUT /identity/v3/projects/79cba51a247e4975a8673bd975625db3/users/519eefa2426f4667a401d71c43001483/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:11.027859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96711c-a288-482d-a511-20f0b9251c82 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:11.033255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-cf96711c-a288-482d-a511-20f0b9251c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:11.111583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:11.172362 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 902/1823] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:11.173726 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188c47c-0c84-44bf-88ed-bf6d445f0631 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:11.174042 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188c47c-0c84-44bf-88ed-bf6d445f0631 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:11.174451 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188c47c-0c84-44bf-88ed-bf6d445f0631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:11.213178 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96711c-a288-482d-a511-20f0b9251c82 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:11.219686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dc86f-94f0-4342-9c74-7efed13fff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:11.220381 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dc86f-94f0-4342-9c74-7efed13fff62 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:11.220867 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dc86f-94f0-4342-9c74-7efed13fff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:11.221308 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dc86f-94f0-4342-9c74-7efed13fff62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:11.222765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dc86f-94f0-4342-9c74-7efed13fff62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:11.224777 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dc86f-94f0-4342-9c74-7efed13fff62 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:11.226321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-c188c47c-0c84-44bf-88ed-bf6d445f0631 None None] MFA Rules not processed for user `519eefa2426f4667a401d71c43001483`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:11.230725 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c188c47c-0c84-44bf-88ed-bf6d445f0631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:11.253383 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 922/1824] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:11.254508 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dc86f-94f0-4342-9c74-7efed13fff62 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:11.272710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dc86f-94f0-4342-9c74-7efed13fff62 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:11.282114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ba645-2d62-4e91-9aa3-aaf98e35e0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:11.282114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ba645-2d62-4e91-9aa3-aaf98e35e0ff admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:11.282114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ba645-2d62-4e91-9aa3-aaf98e35e0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:11.282114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ba645-2d62-4e91-9aa3-aaf98e35e0ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:11.282402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ba645-2d62-4e91-9aa3-aaf98e35e0ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:14:11.282402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ba645-2d62-4e91-9aa3-aaf98e35e0ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:11.283239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ba645-2d62-4e91-9aa3-aaf98e35e0ff admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:11.319584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 923/1825] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:14:11.319584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ba645-2d62-4e91-9aa3-aaf98e35e0ff admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:11.337167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 903/1826] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:11.337167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188c47c-0c84-44bf-88ed-bf6d445f0631 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:11.351922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188c47c-0c84-44bf-88ed-bf6d445f0631 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:11.356601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ba645-2d62-4e91-9aa3-aaf98e35e0ff admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:11.361416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81adee-c90b-438e-adc8-1bb06858b545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:11.362436 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d757a-f3ac-4be6-807b-0ce55a85f463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:11.363396 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d757a-f3ac-4be6-807b-0ce55a85f463 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:11.363690 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d757a-f3ac-4be6-807b-0ce55a85f463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:11.364619 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d757a-f3ac-4be6-807b-0ce55a85f463 admin admin] PATH_INFO: `/v3/projects/55d46b9d2ee54c13afc48f883c648d16/users/a90ce57806964779816f3a0b542f12ef/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:11.364994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81adee-c90b-438e-adc8-1bb06858b545 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:11.365704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81adee-c90b-438e-adc8-1bb06858b545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:11.366749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81adee-c90b-438e-adc8-1bb06858b545 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:11.368211 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d757a-f3ac-4be6-807b-0ce55a85f463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55d46b9d2ee54c13afc48f883c648d16, user_id=a90ce57806964779816f3a0b542f12ef, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:11.369276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81adee-c90b-438e-adc8-1bb06858b545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:11.378315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d757a-f3ac-4be6-807b-0ce55a85f463 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:11.381875 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81adee-c90b-438e-adc8-1bb06858b545 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:11.399847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 924/1827] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:11 2022] PUT /identity/v3/projects/55d46b9d2ee54c13afc48f883c648d16/users/a90ce57806964779816f3a0b542f12ef/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:11.404834 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d757a-f3ac-4be6-807b-0ce55a85f463 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:11.562488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 904/1828] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:11 2022] POST /identity/v3/projects => generated 365 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:11.564280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81adee-c90b-438e-adc8-1bb06858b545 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:11.618562 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d757a-f3ac-4be6-807b-0ce55a85f463 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:11.658268 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:11.748299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81adee-c90b-438e-adc8-1bb06858b545 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:11.761644 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756853b4-ee6e-4253-90dd-67f6f7b17f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:11.762251 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756853b4-ee6e-4253-90dd-67f6f7b17f07 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:11.762670 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756853b4-ee6e-4253-90dd-67f6f7b17f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:11.762853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756853b4-ee6e-4253-90dd-67f6f7b17f07 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:11.763574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756853b4-ee6e-4253-90dd-67f6f7b17f07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:11.768345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4b11a-5e0c-4032-a1c5-9bff43a40664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:11.769046 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4b11a-5e0c-4032-a1c5-9bff43a40664 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:11.769260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4b11a-5e0c-4032-a1c5-9bff43a40664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:11.769461 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4b11a-5e0c-4032-a1c5-9bff43a40664 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:11.771661 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4b11a-5e0c-4032-a1c5-9bff43a40664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:11.771972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756853b4-ee6e-4253-90dd-67f6f7b17f07 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:11.775686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4b11a-5e0c-4032-a1c5-9bff43a40664 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:11.943026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 905/1829] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:11 2022] POST /identity/v3/users => generated 283 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:11.947140 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 925/1830] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:11 2022] POST /identity/v3/projects => generated 377 bytes in 546 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:11.947972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756853b4-ee6e-4253-90dd-67f6f7b17f07 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:11.948871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4b11a-5e0c-4032-a1c5-9bff43a40664 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:11.979539 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756853b4-ee6e-4253-90dd-67f6f7b17f07 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:12.142305 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4b11a-5e0c-4032-a1c5-9bff43a40664 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:12.151320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31feed-6e80-4287-881f-9112b76c0cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:12.152062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31feed-6e80-4287-881f-9112b76c0cbe admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:12.152354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31feed-6e80-4287-881f-9112b76c0cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:12.152640 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31feed-6e80-4287-881f-9112b76c0cbe admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:12.153358 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31feed-6e80-4287-881f-9112b76c0cbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:12.156156 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31feed-6e80-4287-881f-9112b76c0cbe admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:12.160516 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9a3fa-1bb6-4334-a634-0fdfd5a8f46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:12.162715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9a3fa-1bb6-4334-a634-0fdfd5a8f46f admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:12.163323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9a3fa-1bb6-4334-a634-0fdfd5a8f46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:12.163841 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9a3fa-1bb6-4334-a634-0fdfd5a8f46f admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:12.165054 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9a3fa-1bb6-4334-a634-0fdfd5a8f46f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:12.167319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9a3fa-1bb6-4334-a634-0fdfd5a8f46f admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:12.187462 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 906/1831] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:12.188802 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9a3fa-1bb6-4334-a634-0fdfd5a8f46f admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:12.211085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9a3fa-1bb6-4334-a634-0fdfd5a8f46f admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:12.218768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbb7c9-4762-4602-94f3-ff3ec3c63980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:12.220227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbb7c9-4762-4602-94f3-ff3ec3c63980 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:12.220227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbb7c9-4762-4602-94f3-ff3ec3c63980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:12.220416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbb7c9-4762-4602-94f3-ff3ec3c63980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:12.221312 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dbb7c9-4762-4602-94f3-ff3ec3c63980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:12.225777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dbb7c9-4762-4602-94f3-ff3ec3c63980 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:12.248318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 907/1832] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:12.249491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbb7c9-4762-4602-94f3-ff3ec3c63980 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:12.263418 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbb7c9-4762-4602-94f3-ff3ec3c63980 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:12.267188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 926/1833] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:11 2022] POST /identity/v3/users => generated 289 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:12.270108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae30f3-c0d7-409d-84eb-559ada820986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:12.271020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31feed-6e80-4287-881f-9112b76c0cbe admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:12.271179 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae30f3-c0d7-409d-84eb-559ada820986 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:12.271478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae30f3-c0d7-409d-84eb-559ada820986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:12.271765 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae30f3-c0d7-409d-84eb-559ada820986 admin admin] PATH_INFO: `/v3/projects/c89a4abfc10747478072af1a662da4b6/users/e70279b0cd7242058d70d2e9dec4666a/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:12.284723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:12.294604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae30f3-c0d7-409d-84eb-559ada820986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89a4abfc10747478072af1a662da4b6, user_id=e70279b0cd7242058d70d2e9dec4666a, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:12.297759 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae30f3-c0d7-409d-84eb-559ada820986 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:12.323269 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31feed-6e80-4287-881f-9112b76c0cbe admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:12.339550 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531806ea-e348-42f9-bed8-84bf6ba5f1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:12.340267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531806ea-e348-42f9-bed8-84bf6ba5f1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:12.343152 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531806ea-e348-42f9-bed8-84bf6ba5f1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:12.344376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531806ea-e348-42f9-bed8-84bf6ba5f1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:12.348382 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531806ea-e348-42f9-bed8-84bf6ba5f1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:12.354882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531806ea-e348-42f9-bed8-84bf6ba5f1f1 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:12.375287 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 927/1834] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:12.379519 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531806ea-e348-42f9-bed8-84bf6ba5f1f1 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:12.415851 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 908/1835] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:12 2022] PUT /identity/v3/projects/c89a4abfc10747478072af1a662da4b6/users/e70279b0cd7242058d70d2e9dec4666a/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:12.429701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28bcbf-f419-4171-85ff-dd8a8c4a10db None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:12.430023 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28bcbf-f419-4171-85ff-dd8a8c4a10db None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:12.430277 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28bcbf-f419-4171-85ff-dd8a8c4a10db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:12.463474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531806ea-e348-42f9-bed8-84bf6ba5f1f1 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:12.511366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-7d28bcbf-f419-4171-85ff-dd8a8c4a10db None None] MFA Rules not processed for user `e70279b0cd7242058d70d2e9dec4666a`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:12.512887 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7d28bcbf-f419-4171-85ff-dd8a8c4a10db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:12.664715 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3de607-940f-400e-b45d-3024b74e6475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:12.665862 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3de607-940f-400e-b45d-3024b74e6475 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:12.666373 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3de607-940f-400e-b45d-3024b74e6475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:12.666931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3de607-940f-400e-b45d-3024b74e6475 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:12.667905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3de607-940f-400e-b45d-3024b74e6475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:12.669829 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3de607-940f-400e-b45d-3024b74e6475 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:12.691988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 928/1836] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:12.692260 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 909/1837] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:14:12.692377 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3de607-940f-400e-b45d-3024b74e6475 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:12.694293 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28bcbf-f419-4171-85ff-dd8a8c4a10db None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:12.710578 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3de607-940f-400e-b45d-3024b74e6475 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:12.717924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28bcbf-f419-4171-85ff-dd8a8c4a10db None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:12.732039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22db67-5ee3-453e-a2c2-a890fee33dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:12.733386 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22db67-5ee3-453e-a2c2-a890fee33dfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:12.734008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22db67-5ee3-453e-a2c2-a890fee33dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:12.734486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22db67-5ee3-453e-a2c2-a890fee33dfd admin admin] PATH_INFO: `/v3/projects/da073f59c3d14771a474e729551691f9/users/4beea81dcac641edb00a51c2682f7c20/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:12.745574 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e945d-dcf1-4070-8bd6-2395cc995f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:12.751790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e945d-dcf1-4070-8bd6-2395cc995f29 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:12.751790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e945d-dcf1-4070-8bd6-2395cc995f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:12.751790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e945d-dcf1-4070-8bd6-2395cc995f29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:12.751790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e945d-dcf1-4070-8bd6-2395cc995f29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:12.752903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e945d-dcf1-4070-8bd6-2395cc995f29 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:12.776617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22db67-5ee3-453e-a2c2-a890fee33dfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=da073f59c3d14771a474e729551691f9, user_id=4beea81dcac641edb00a51c2682f7c20, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:12.783685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22db67-5ee3-453e-a2c2-a890fee33dfd admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:12.923745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 910/1838] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:12 2022] PUT /identity/v3/projects/da073f59c3d14771a474e729551691f9/users/4beea81dcac641edb00a51c2682f7c20/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:12.939496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138ae11-866a-4a33-bad0-4a9cbc7105c5 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:12.939907 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138ae11-866a-4a33-bad0-4a9cbc7105c5 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:12.940262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138ae11-866a-4a33-bad0-4a9cbc7105c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:12.968012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 929/1839] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:12 2022] POST /identity/v3/projects => generated 365 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:12.971045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e945d-dcf1-4070-8bd6-2395cc995f29 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:12.992546 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-3138ae11-866a-4a33-bad0-4a9cbc7105c5 None None] MFA Rules not processed for user `4beea81dcac641edb00a51c2682f7c20`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:12.996582 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-3138ae11-866a-4a33-bad0-4a9cbc7105c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:13.124808 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 911/1840] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:14:13.129987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138ae11-866a-4a33-bad0-4a9cbc7105c5 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:13.217911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e945d-dcf1-4070-8bd6-2395cc995f29 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:13.227453 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f405e5f-cbae-46a2-9f0f-c7de4dbfccb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:13.227696 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f405e5f-cbae-46a2-9f0f-c7de4dbfccb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:13.228129 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f405e5f-cbae-46a2-9f0f-c7de4dbfccb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:13.228432 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f405e5f-cbae-46a2-9f0f-c7de4dbfccb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:13.230146 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f405e5f-cbae-46a2-9f0f-c7de4dbfccb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:13.231333 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f405e5f-cbae-46a2-9f0f-c7de4dbfccb7 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:13.231783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138ae11-866a-4a33-bad0-4a9cbc7105c5 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:13.240906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2192bc6-2d55-401b-b69a-5b28a6e25f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:13.241849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2192bc6-2d55-401b-b69a-5b28a6e25f4d admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:13.241849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2192bc6-2d55-401b-b69a-5b28a6e25f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:13.241849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2192bc6-2d55-401b-b69a-5b28a6e25f4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:13.243959 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2192bc6-2d55-401b-b69a-5b28a6e25f4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:13.245379 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2192bc6-2d55-401b-b69a-5b28a6e25f4d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:13.432962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 912/1841] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:13 2022] POST /identity/v3/projects => generated 377 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:14:13.434394 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2192bc6-2d55-401b-b69a-5b28a6e25f4d admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:13.436907 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 930/1842] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:12 2022] POST /identity/v3/users => generated 289 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:13.442636 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f405e5f-cbae-46a2-9f0f-c7de4dbfccb7 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:13.459021 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f405e5f-cbae-46a2-9f0f-c7de4dbfccb7 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:13.547782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2192bc6-2d55-401b-b69a-5b28a6e25f4d admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:13.558592 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3ca17-70c2-4109-8ad6-7b22f7a2e98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:13.559448 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3ca17-70c2-4109-8ad6-7b22f7a2e98c admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:13.559904 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3ca17-70c2-4109-8ad6-7b22f7a2e98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:13.560318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3ca17-70c2-4109-8ad6-7b22f7a2e98c admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:13.561338 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3ca17-70c2-4109-8ad6-7b22f7a2e98c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:13.563804 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3ca17-70c2-4109-8ad6-7b22f7a2e98c admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:13.621880 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0c124-2f11-4177-a4bc-e8e67934effb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:13.623348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0c124-2f11-4177-a4bc-e8e67934effb admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:13.623957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0c124-2f11-4177-a4bc-e8e67934effb admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:13.624565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0c124-2f11-4177-a4bc-e8e67934effb admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:13.626112 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0c124-2f11-4177-a4bc-e8e67934effb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:13.628716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0c124-2f11-4177-a4bc-e8e67934effb admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:13.648722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 931/1843] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:13.653066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0c124-2f11-4177-a4bc-e8e67934effb admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:13.675761 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 913/1844] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:13 2022] POST /identity/v3/users => generated 295 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:13.680710 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3ca17-70c2-4109-8ad6-7b22f7a2e98c admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:13.695937 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3ca17-70c2-4109-8ad6-7b22f7a2e98c admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:13.700566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0c124-2f11-4177-a4bc-e8e67934effb admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:13.701836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7776a81-9c79-4cc0-a773-bc3f4ff3d208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:13.702716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7776a81-9c79-4cc0-a773-bc3f4ff3d208 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:13.703590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7776a81-9c79-4cc0-a773-bc3f4ff3d208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:13.703960 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7776a81-9c79-4cc0-a773-bc3f4ff3d208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:13.704742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7776a81-9c79-4cc0-a773-bc3f4ff3d208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:13.706519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7776a81-9c79-4cc0-a773-bc3f4ff3d208 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:13.720132 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb6f2b-2843-41c0-b2e3-068aa2ca6ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:13.721737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb6f2b-2843-41c0-b2e3-068aa2ca6ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:13.722029 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb6f2b-2843-41c0-b2e3-068aa2ca6ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:13.722313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb6f2b-2843-41c0-b2e3-068aa2ca6ac3 admin admin] PATH_INFO: `/v3/projects/cee52315be764f5d8741086091a28aed/users/506f816cd9624e51bb55872bbf9747ad/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:13.724766 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 914/1845] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:14:13.729488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7776a81-9c79-4cc0-a773-bc3f4ff3d208 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:13.743534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7776a81-9c79-4cc0-a773-bc3f4ff3d208 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:13.748753 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b39407-d899-4b34-aa6c-eaff108c2bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:13.750222 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b39407-d899-4b34-aa6c-eaff108c2bda admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:13.750573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b39407-d899-4b34-aa6c-eaff108c2bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:13.750846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b39407-d899-4b34-aa6c-eaff108c2bda admin admin] PATH_INFO: `/v3/projects/16bc9b6f03d442e9a6e0c0d8a6712a9f/users/c106703e498b404bae9f242482f23589/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:13.751804 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb6f2b-2843-41c0-b2e3-068aa2ca6ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee52315be764f5d8741086091a28aed, user_id=506f816cd9624e51bb55872bbf9747ad, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:13.755800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb6f2b-2843-41c0-b2e3-068aa2ca6ac3 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:13.773715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b39407-d899-4b34-aa6c-eaff108c2bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=16bc9b6f03d442e9a6e0c0d8a6712a9f, user_id=c106703e498b404bae9f242482f23589, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:13.778272 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b39407-d899-4b34-aa6c-eaff108c2bda admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:13.837380 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:13.849758 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 932/1846] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:13 2022] PUT /identity/v3/projects/cee52315be764f5d8741086091a28aed/users/506f816cd9624e51bb55872bbf9747ad/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:14:13.854118 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb6f2b-2843-41c0-b2e3-068aa2ca6ac3 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:13.919497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 915/1847] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:13 2022] PUT /identity/v3/projects/16bc9b6f03d442e9a6e0c0d8a6712a9f/users/c106703e498b404bae9f242482f23589/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:13.923750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b39407-d899-4b34-aa6c-eaff108c2bda admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:13.966171 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:13.969236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb6f2b-2843-41c0-b2e3-068aa2ca6ac3 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:14.009782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b39407-d899-4b34-aa6c-eaff108c2bda admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:14.014983 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17c519-6893-48fd-b5ba-a6b5d59c011e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:14.015672 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17c519-6893-48fd-b5ba-a6b5d59c011e admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:14.015963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17c519-6893-48fd-b5ba-a6b5d59c011e admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:14.016243 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17c519-6893-48fd-b5ba-a6b5d59c011e admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:14.016983 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17c519-6893-48fd-b5ba-a6b5d59c011e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:14.018896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17c519-6893-48fd-b5ba-a6b5d59c011e admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:14.033189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 916/1848] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:14:14.038261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17c519-6893-48fd-b5ba-a6b5d59c011e admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:14.042007 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e7d94-3900-42ce-bb4e-8792741814db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:14.042734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e7d94-3900-42ce-bb4e-8792741814db admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:14.043021 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e7d94-3900-42ce-bb4e-8792741814db admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:14.043299 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e7d94-3900-42ce-bb4e-8792741814db admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:14.044030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e7d94-3900-42ce-bb4e-8792741814db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:14.045671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e7d94-3900-42ce-bb4e-8792741814db admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:14.053194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17c519-6893-48fd-b5ba-a6b5d59c011e admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:14.057615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 933/1849] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:14.060159 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23f2e1-4579-4ca1-a369-60ee3d7f44cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:14.061216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23f2e1-4579-4ca1-a369-60ee3d7f44cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:14.061488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23f2e1-4579-4ca1-a369-60ee3d7f44cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:14.061911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23f2e1-4579-4ca1-a369-60ee3d7f44cf admin admin] PATH_INFO: `/v3/projects/16bc9b6f03d442e9a6e0c0d8a6712a9f/users/c106703e498b404bae9f242482f23589/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:14.063837 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e7d94-3900-42ce-bb4e-8792741814db admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:14.064337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23f2e1-4579-4ca1-a369-60ee3d7f44cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=16bc9b6f03d442e9a6e0c0d8a6712a9f, user_id=c106703e498b404bae9f242482f23589, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:14.068130 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23f2e1-4579-4ca1-a369-60ee3d7f44cf admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:14.076881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e7d94-3900-42ce-bb4e-8792741814db admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:14.080914 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 917/1850] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:14 2022] PUT /identity/v3/projects/16bc9b6f03d442e9a6e0c0d8a6712a9f/users/c106703e498b404bae9f242482f23589/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:14:14.092086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba44e27-5f19-41cc-aea6-6fea304608fd None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:14.092433 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba44e27-5f19-41cc-aea6-6fea304608fd None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:14.092638 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba44e27-5f19-41cc-aea6-6fea304608fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:14.127131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-cba44e27-5f19-41cc-aea6-6fea304608fd None None] MFA Rules not processed for user `c106703e498b404bae9f242482f23589`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:14.128605 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-cba44e27-5f19-41cc-aea6-6fea304608fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:14.154986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a5e7b-734f-4f5b-adef-47b83609b170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:14.155889 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a5e7b-734f-4f5b-adef-47b83609b170 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:14.156178 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a5e7b-734f-4f5b-adef-47b83609b170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:14.156535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a5e7b-734f-4f5b-adef-47b83609b170 admin admin] PATH_INFO: `/v3/projects/cee52315be764f5d8741086091a28aed/users/506f816cd9624e51bb55872bbf9747ad/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:14.158589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a5e7b-734f-4f5b-adef-47b83609b170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee52315be764f5d8741086091a28aed, user_id=506f816cd9624e51bb55872bbf9747ad, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:14.161568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a5e7b-734f-4f5b-adef-47b83609b170 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:14.172987 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 934/1851] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:14 2022] PUT /identity/v3/projects/cee52315be764f5d8741086091a28aed/users/506f816cd9624e51bb55872bbf9747ad/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:14.184704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c399a4-38b9-4bad-8a9c-0f65f1d982ee None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:14.185285 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c399a4-38b9-4bad-8a9c-0f65f1d982ee None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:14.185573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c399a4-38b9-4bad-8a9c-0f65f1d982ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:14.211242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 918/1852] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:14:14.223325 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba44e27-5f19-41cc-aea6-6fea304608fd None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:14.225344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-cba44e27-5f19-41cc-aea6-6fea304608fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:14.227579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-88c399a4-38b9-4bad-8a9c-0f65f1d982ee None None] MFA Rules not processed for user `506f816cd9624e51bb55872bbf9747ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:14.228885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-88c399a4-38b9-4bad-8a9c-0f65f1d982ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:14.291952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba44e27-5f19-41cc-aea6-6fea304608fd None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:14.352522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 935/1853] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:14:14.368614 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c399a4-38b9-4bad-8a9c-0f65f1d982ee None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:14.371266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-88c399a4-38b9-4bad-8a9c-0f65f1d982ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:14.376374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a4664-5afb-4cce-a6b9-b01d2cd2f3ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:14.379064 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a4664-5afb-4cce-a6b9-b01d2cd2f3ce service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:14.379064 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a4664-5afb-4cce-a6b9-b01d2cd2f3ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:14.379064 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a4664-5afb-4cce-a6b9-b01d2cd2f3ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:14.393340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c399a4-38b9-4bad-8a9c-0f65f1d982ee None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:14.401968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbf90a-9f7d-47f5-835a-d6262e259deb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:14.401968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbf90a-9f7d-47f5-835a-d6262e259deb service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:14.401968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbf90a-9f7d-47f5-835a-d6262e259deb service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:14.401968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbf90a-9f7d-47f5-835a-d6262e259deb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:14.415441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbf90a-9f7d-47f5-835a-d6262e259deb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:14.419326 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbf90a-9f7d-47f5-835a-d6262e259deb service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:14.443641 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 936/1854] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:14.446169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f55b76-ab0c-4ada-8ed5-48173c098c57 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:14.450262 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f55b76-ab0c-4ada-8ed5-48173c098c57 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:14.451310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f55b76-ab0c-4ada-8ed5-48173c098c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:14.502241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-f5f55b76-ab0c-4ada-8ed5-48173c098c57 None None] MFA Rules not processed for user `a90ce57806964779816f3a0b542f12ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:14.503656 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f5f55b76-ab0c-4ada-8ed5-48173c098c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:14.544879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a4664-5afb-4cce-a6b9-b01d2cd2f3ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:14.546351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a4664-5afb-4cce-a6b9-b01d2cd2f3ce service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:14.563336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 919/1855] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:14.604613 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 937/1856] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:14.721471 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a4664-5afb-4cce-a6b9-b01d2cd2f3ce service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:14.732181 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a4664-5afb-4cce-a6b9-b01d2cd2f3ce service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:14.738340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63e9d6-9cc2-4d56-ac5e-7cf5ada5e26f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:14.739946 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63e9d6-9cc2-4d56-ac5e-7cf5ada5e26f service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:14.740307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63e9d6-9cc2-4d56-ac5e-7cf5ada5e26f service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:14.740647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63e9d6-9cc2-4d56-ac5e-7cf5ada5e26f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:14.742889 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ac63e9d6-9cc2-4d56-ac5e-7cf5ada5e26f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:14.764554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f55b76-ab0c-4ada-8ed5-48173c098c57 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:14.777175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f55b76-ab0c-4ada-8ed5-48173c098c57 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:14.781893 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4509fd-7b33-42ca-b794-35679ae9504b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:14.782255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4509fd-7b33-42ca-b794-35679ae9504b service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:14.782528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4509fd-7b33-42ca-b794-35679ae9504b service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:14.782842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4509fd-7b33-42ca-b794-35679ae9504b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:14.838300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac63e9d6-9cc2-4d56-ac5e-7cf5ada5e26f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:14.839527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac63e9d6-9cc2-4d56-ac5e-7cf5ada5e26f service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:14.852844 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 920/1857] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:14.871525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4509fd-7b33-42ca-b794-35679ae9504b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:14.871525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4509fd-7b33-42ca-b794-35679ae9504b service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:14.882740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 938/1858] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:14:15.676300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63e9d6-9cc2-4d56-ac5e-7cf5ada5e26f service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:15.685709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:15.691930 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63e9d6-9cc2-4d56-ac5e-7cf5ada5e26f service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:15.696617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c2231-4860-42f6-84fd-83e3bae045a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:15.697329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c2231-4860-42f6-84fd-83e3bae045a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:15.697613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c2231-4860-42f6-84fd-83e3bae045a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:15.697900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c2231-4860-42f6-84fd-83e3bae045a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:15.761720 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c2231-4860-42f6-84fd-83e3bae045a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:15.763219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c2231-4860-42f6-84fd-83e3bae045a3 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:15.774766 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 921/1859] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:16.763630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:17.055655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4509fd-7b33-42ca-b794-35679ae9504b service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:17.065009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4509fd-7b33-42ca-b794-35679ae9504b service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:17.070413 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4864bc-fc3d-4258-9bcc-cef6e1c0d556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:17.071136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4864bc-fc3d-4258-9bcc-cef6e1c0d556 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:17.071421 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4864bc-fc3d-4258-9bcc-cef6e1c0d556 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:17.071701 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4864bc-fc3d-4258-9bcc-cef6e1c0d556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:17.082198 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4864bc-fc3d-4258-9bcc-cef6e1c0d556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:17.085302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4864bc-fc3d-4258-9bcc-cef6e1c0d556 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:17.101252 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 939/1860] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:17.432327 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:18.485424 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f70d1c-a745-4e1a-9bc3-e30474bc614c None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:18.485424 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f70d1c-a745-4e1a-9bc3-e30474bc614c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:18.486211 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f70d1c-a745-4e1a-9bc3-e30474bc614c None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:18.487683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 922/1861] 10.222.0.83 () {54 vars in 921 bytes} [Mon Jul 18 15:14:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:14:18.499543 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16e3b0-ade5-4c8f-96dc-1c1a7eec13cb None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:18.500285 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16e3b0-ade5-4c8f-96dc-1c1a7eec13cb None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:18.500581 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16e3b0-ade5-4c8f-96dc-1c1a7eec13cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:18.523496 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-cf16e3b0-ade5-4c8f-96dc-1c1a7eec13cb None None] MFA Rules not processed for user `7e1355d18fb049468651b3675c217d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:18.524738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-cf16e3b0-ade5-4c8f-96dc-1c1a7eec13cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:18.596402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 940/1862] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Jul 18 15:14:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:14:18.597678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:18.613546 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f70d1c-a745-4e1a-9bc3-e30474bc614c None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:18.615419 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b4f70d1c-a745-4e1a-9bc3-e30474bc614c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:18.634479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f70d1c-a745-4e1a-9bc3-e30474bc614c None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:18.640465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9058e90-2f06-44d2-a12f-3c61d2e2eb3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:18.641249 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9058e90-2f06-44d2-a12f-3c61d2e2eb3c service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:18.641528 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9058e90-2f06-44d2-a12f-3c61d2e2eb3c service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:18.641814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9058e90-2f06-44d2-a12f-3c61d2e2eb3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:18.653211 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9058e90-2f06-44d2-a12f-3c61d2e2eb3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:18.654874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9058e90-2f06-44d2-a12f-3c61d2e2eb3c service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:18.676682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 923/1863] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:19.251805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:20.129486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:21.863607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16e3b0-ade5-4c8f-96dc-1c1a7eec13cb None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:21.922911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9058e90-2f06-44d2-a12f-3c61d2e2eb3c service nova] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:21.926382 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c9058e90-2f06-44d2-a12f-3c61d2e2eb3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:21.984603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16e3b0-ade5-4c8f-96dc-1c1a7eec13cb None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:21.993239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bfe29-10f2-4d0f-b8c8-2045470be118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:21.994408 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bfe29-10f2-4d0f-b8c8-2045470be118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:21.994817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bfe29-10f2-4d0f-b8c8-2045470be118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:21.995107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bfe29-10f2-4d0f-b8c8-2045470be118 admin admin] PATH_INFO: `/v3/users/519eefa2426f4667a401d71c43001483` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:21.996371 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bfe29-10f2-4d0f-b8c8-2045470be118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=519eefa2426f4667a401d71c43001483)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:21.998404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bfe29-10f2-4d0f-b8c8-2045470be118 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:22.035138 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9058e90-2f06-44d2-a12f-3c61d2e2eb3c service nova] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:22.040191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d518caf-b200-4473-b1f0-30d9c04f9e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:22.041089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d518caf-b200-4473-b1f0-30d9c04f9e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:22.041404 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d518caf-b200-4473-b1f0-30d9c04f9e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:22.041690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d518caf-b200-4473-b1f0-30d9c04f9e87 admin admin] PATH_INFO: `/v3/users/ab4e8ed13a79403e8397840370abc1d9` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:22.042984 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d518caf-b200-4473-b1f0-30d9c04f9e87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab4e8ed13a79403e8397840370abc1d9)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:22.052888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d518caf-b200-4473-b1f0-30d9c04f9e87 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:22.052888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:22.264088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f06bfe29-10f2-4d0f-b8c8-2045470be118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '519eefa2426f4667a401d71c43001483'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:22.273461 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f06bfe29-10f2-4d0f-b8c8-2045470be118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '519eefa2426f4667a401d71c43001483'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:22.274240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f06bfe29-10f2-4d0f-b8c8-2045470be118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '519eefa2426f4667a401d71c43001483'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:22.303953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f06bfe29-10f2-4d0f-b8c8-2045470be118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '519eefa2426f4667a401d71c43001483'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:22.325166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-2d518caf-b200-4473-b1f0-30d9c04f9e87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab4e8ed13a79403e8397840370abc1d9'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:22.326567 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-2d518caf-b200-4473-b1f0-30d9c04f9e87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab4e8ed13a79403e8397840370abc1d9'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:22.327631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-2d518caf-b200-4473-b1f0-30d9c04f9e87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab4e8ed13a79403e8397840370abc1d9'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:22.381084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f06bfe29-10f2-4d0f-b8c8-2045470be118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '519eefa2426f4667a401d71c43001483'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:22.388543 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 941/1864] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:21 2022] DELETE /identity/v3/users/519eefa2426f4667a401d71c43001483 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:22.393378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862f5d7-2c24-4a49-ba03-bcf8fd95b829 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:22.393673 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862f5d7-2c24-4a49-ba03-bcf8fd95b829 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:22.393957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862f5d7-2c24-4a49-ba03-bcf8fd95b829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:22.420745 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-c862f5d7-2c24-4a49-ba03-bcf8fd95b829 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:22.422007 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c862f5d7-2c24-4a49-ba03-bcf8fd95b829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:22.443700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-2d518caf-b200-4473-b1f0-30d9c04f9e87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab4e8ed13a79403e8397840370abc1d9'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:22.470836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-2d518caf-b200-4473-b1f0-30d9c04f9e87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab4e8ed13a79403e8397840370abc1d9'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:22.493351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 924/1865] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:21 2022] DELETE /identity/v3/users/ab4e8ed13a79403e8397840370abc1d9 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:22.502403 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fc82d-ba6f-49a2-9773-31e028a2aa37 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:22.503105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fc82d-ba6f-49a2-9773-31e028a2aa37 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:22.503766 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fc82d-ba6f-49a2-9773-31e028a2aa37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:22.506612 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 942/1866] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:14:22.518301 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c862f5d7-2c24-4a49-ba03-bcf8fd95b829 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:22.519040 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c862f5d7-2c24-4a49-ba03-bcf8fd95b829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:22.538259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-293fc82d-ba6f-49a2-9773-31e028a2aa37 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:22.540682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-293fc82d-ba6f-49a2-9773-31e028a2aa37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:22.547109 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:22.597702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c862f5d7-2c24-4a49-ba03-bcf8fd95b829 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:22.659088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a557a7-de1d-450d-af1b-0c1737556451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:22.659770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a557a7-de1d-450d-af1b-0c1737556451 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:22.660063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a557a7-de1d-450d-af1b-0c1737556451 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:22.660348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a557a7-de1d-450d-af1b-0c1737556451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:22.671970 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 925/1867] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:22.677944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a557a7-de1d-450d-af1b-0c1737556451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:22.682196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a557a7-de1d-450d-af1b-0c1737556451 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:22.685523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fc82d-ba6f-49a2-9773-31e028a2aa37 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:22.703145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 943/1868] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:14:22.705607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fc82d-ba6f-49a2-9773-31e028a2aa37 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:22.713535 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03561f6-7f16-4296-8917-5e95b0dc075f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:22.714610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03561f6-7f16-4296-8917-5e95b0dc075f service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:22.714990 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03561f6-7f16-4296-8917-5e95b0dc075f service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:22.715353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03561f6-7f16-4296-8917-5e95b0dc075f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:22.733901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03561f6-7f16-4296-8917-5e95b0dc075f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:22.738600 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03561f6-7f16-4296-8917-5e95b0dc075f service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:22.754806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 926/1869] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:14:22.814731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:23.406347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a557a7-de1d-450d-af1b-0c1737556451 service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:23.408054 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-32a557a7-de1d-450d-af1b-0c1737556451 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:23.422475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a557a7-de1d-450d-af1b-0c1737556451 service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:23.428274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:23.429184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:23.429472 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:23.429957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846 admin admin] PATH_INFO: `/v3/projects/79cba51a247e4975a8673bd975625db3` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:23.431136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79cba51a247e4975a8673bd975625db3)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:23.433694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:23.477172 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03561f6-7f16-4296-8917-5e95b0dc075f service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:23.479161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d03561f6-7f16-4296-8917-5e95b0dc075f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:23.494478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03561f6-7f16-4296-8917-5e95b0dc075f service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:23.500522 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ecb061-6c1b-4d12-bba9-62ae500117f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:23.501233 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecb061-6c1b-4d12-bba9-62ae500117f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:23.501523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecb061-6c1b-4d12-bba9-62ae500117f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:23.501810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecb061-6c1b-4d12-bba9-62ae500117f7 admin admin] PATH_INFO: `/v3/projects/184452ed59424f808208e40117401e90` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:23.503245 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ecb061-6c1b-4d12-bba9-62ae500117f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=184452ed59424f808208e40117401e90)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:23.505821 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ecb061-6c1b-4d12-bba9-62ae500117f7 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:23.565773 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79cba51a247e4975a8673bd975625db3', 'request_id': 'req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:23.573009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79cba51a247e4975a8673bd975625db3', 'request_id': 'req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:23.679209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c7ecb061-6c1b-4d12-bba9-62ae500117f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '184452ed59424f808208e40117401e90', 'request_id': 'req-c7ecb061-6c1b-4d12-bba9-62ae500117f7'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:23.690306 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79cba51a247e4975a8673bd975625db3', 'request_id': 'req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:23.693074 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c7ecb061-6c1b-4d12-bba9-62ae500117f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '184452ed59424f808208e40117401e90', 'request_id': 'req-c7ecb061-6c1b-4d12-bba9-62ae500117f7'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:23.699604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c7ecb061-6c1b-4d12-bba9-62ae500117f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '184452ed59424f808208e40117401e90', 'request_id': 'req-c7ecb061-6c1b-4d12-bba9-62ae500117f7'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:23.702618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:23.703667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846 admin admin] The token cache is being invalidate because project 79cba51a247e4975a8673bd975625db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:23.703972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:23.704691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:23.706338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 944/1870] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:23 2022] DELETE /identity/v3/projects/79cba51a247e4975a8673bd975625db3 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:23.712724 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:23.714618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:23.788900 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e98237-9bf1-4ccc-8fa4-3df82dc3c846 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:23.850988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835bae4-ff36-4e3a-9e75-44653a4259f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:23.851882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835bae4-ff36-4e3a-9e75-44653a4259f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:23.852170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835bae4-ff36-4e3a-9e75-44653a4259f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:23.852456 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835bae4-ff36-4e3a-9e75-44653a4259f1 admin admin] PATH_INFO: `/v3/users/4beea81dcac641edb00a51c2682f7c20` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:23.853784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835bae4-ff36-4e3a-9e75-44653a4259f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4beea81dcac641edb00a51c2682f7c20)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:23.855803 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835bae4-ff36-4e3a-9e75-44653a4259f1 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:23.861220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c7ecb061-6c1b-4d12-bba9-62ae500117f7 admin admin] The token cache is being invalidate because project 184452ed59424f808208e40117401e90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:23.861783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c7ecb061-6c1b-4d12-bba9-62ae500117f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:23.862781 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c7ecb061-6c1b-4d12-bba9-62ae500117f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:23.866486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 927/1871] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:23 2022] DELETE /identity/v3/projects/184452ed59424f808208e40117401e90 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:14:23.871291 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ecb061-6c1b-4d12-bba9-62ae500117f7 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:23.873355 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c7ecb061-6c1b-4d12-bba9-62ae500117f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:23.891485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ecb061-6c1b-4d12-bba9-62ae500117f7 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:23.896510 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd681429-b74d-4e64-a62e-4234dd29e5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:23.897431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd681429-b74d-4e64-a62e-4234dd29e5f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:23.897708 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd681429-b74d-4e64-a62e-4234dd29e5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:23.897993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd681429-b74d-4e64-a62e-4234dd29e5f2 admin admin] PATH_INFO: `/v3/users/8f40970abb8d4d109d2368e7fe1cc63b` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:23.899286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd681429-b74d-4e64-a62e-4234dd29e5f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f40970abb8d4d109d2368e7fe1cc63b)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:23.901278 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd681429-b74d-4e64-a62e-4234dd29e5f2 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:24.114498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e835bae4-ff36-4e3a-9e75-44653a4259f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4beea81dcac641edb00a51c2682f7c20'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:24.132713 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e835bae4-ff36-4e3a-9e75-44653a4259f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4beea81dcac641edb00a51c2682f7c20'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:24.134103 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e835bae4-ff36-4e3a-9e75-44653a4259f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4beea81dcac641edb00a51c2682f7c20'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:24.154723 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e835bae4-ff36-4e3a-9e75-44653a4259f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4beea81dcac641edb00a51c2682f7c20'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:24.213868 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-bd681429-b74d-4e64-a62e-4234dd29e5f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f40970abb8d4d109d2368e7fe1cc63b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:24.215018 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-bd681429-b74d-4e64-a62e-4234dd29e5f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f40970abb8d4d109d2368e7fe1cc63b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:24.215753 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-bd681429-b74d-4e64-a62e-4234dd29e5f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f40970abb8d4d109d2368e7fe1cc63b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:24.232341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e835bae4-ff36-4e3a-9e75-44653a4259f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4beea81dcac641edb00a51c2682f7c20'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:24.239391 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 945/1872] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:23 2022] DELETE /identity/v3/users/4beea81dcac641edb00a51c2682f7c20 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:24.352915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-bd681429-b74d-4e64-a62e-4234dd29e5f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f40970abb8d4d109d2368e7fe1cc63b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:24.369192 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-bd681429-b74d-4e64-a62e-4234dd29e5f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f40970abb8d4d109d2368e7fe1cc63b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:24.385788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 928/1873] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:23 2022] DELETE /identity/v3/users/8f40970abb8d4d109d2368e7fe1cc63b => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:14:24.435004 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:24.768701 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835bae4-ff36-4e3a-9e75-44653a4259f1 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:24.769867 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e835bae4-ff36-4e3a-9e75-44653a4259f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:24.862062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835bae4-ff36-4e3a-9e75-44653a4259f1 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:24.914900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd681429-b74d-4e64-a62e-4234dd29e5f2 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:24.916492 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-bd681429-b74d-4e64-a62e-4234dd29e5f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:24.994383 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d732b-0e40-4629-b60b-b5a55bb44f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:24.995327 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d732b-0e40-4629-b60b-b5a55bb44f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:24.995700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d732b-0e40-4629-b60b-b5a55bb44f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:24.996047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d732b-0e40-4629-b60b-b5a55bb44f51 admin admin] PATH_INFO: `/v3/projects/da073f59c3d14771a474e729551691f9` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:24.998036 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d732b-0e40-4629-b60b-b5a55bb44f51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da073f59c3d14771a474e729551691f9)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:25.000136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d732b-0e40-4629-b60b-b5a55bb44f51 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:25.029214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd681429-b74d-4e64-a62e-4234dd29e5f2 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:25.044188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa23bd-122e-4e3c-a338-02140b5b481c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:25.045568 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa23bd-122e-4e3c-a338-02140b5b481c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:25.046111 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa23bd-122e-4e3c-a338-02140b5b481c admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:25.046658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa23bd-122e-4e3c-a338-02140b5b481c admin admin] PATH_INFO: `/v3/projects/cc64eae755244c80bbde04f6acf3bd2b` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:25.048235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa23bd-122e-4e3c-a338-02140b5b481c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc64eae755244c80bbde04f6acf3bd2b)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:25.051143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa23bd-122e-4e3c-a338-02140b5b481c admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:25.181047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-552d732b-0e40-4629-b60b-b5a55bb44f51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da073f59c3d14771a474e729551691f9', 'request_id': 'req-552d732b-0e40-4629-b60b-b5a55bb44f51'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:25.192239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-552d732b-0e40-4629-b60b-b5a55bb44f51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da073f59c3d14771a474e729551691f9', 'request_id': 'req-552d732b-0e40-4629-b60b-b5a55bb44f51'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:25.215048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-15aa23bd-122e-4e3c-a338-02140b5b481c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc64eae755244c80bbde04f6acf3bd2b', 'request_id': 'req-15aa23bd-122e-4e3c-a338-02140b5b481c'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:25.246569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-15aa23bd-122e-4e3c-a338-02140b5b481c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc64eae755244c80bbde04f6acf3bd2b', 'request_id': 'req-15aa23bd-122e-4e3c-a338-02140b5b481c'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:25.266162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-15aa23bd-122e-4e3c-a338-02140b5b481c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc64eae755244c80bbde04f6acf3bd2b', 'request_id': 'req-15aa23bd-122e-4e3c-a338-02140b5b481c'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:25.301703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-552d732b-0e40-4629-b60b-b5a55bb44f51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da073f59c3d14771a474e729551691f9', 'request_id': 'req-552d732b-0e40-4629-b60b-b5a55bb44f51'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:25.328449 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-552d732b-0e40-4629-b60b-b5a55bb44f51 admin admin] The token cache is being invalidate because project da073f59c3d14771a474e729551691f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:25.335615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-552d732b-0e40-4629-b60b-b5a55bb44f51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:25.335615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-552d732b-0e40-4629-b60b-b5a55bb44f51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:25.340182 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 946/1874] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:24 2022] DELETE /identity/v3/projects/da073f59c3d14771a474e729551691f9 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:25.341701 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d732b-0e40-4629-b60b-b5a55bb44f51 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:25.343601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-552d732b-0e40-4629-b60b-b5a55bb44f51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:25.387308 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-15aa23bd-122e-4e3c-a338-02140b5b481c admin admin] The token cache is being invalidate because project cc64eae755244c80bbde04f6acf3bd2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:25.387637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-15aa23bd-122e-4e3c-a338-02140b5b481c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:25.393138 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-15aa23bd-122e-4e3c-a338-02140b5b481c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:25.395042 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 929/1875] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:24 2022] DELETE /identity/v3/projects/cc64eae755244c80bbde04f6acf3bd2b => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:14:25.397075 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e55f14-921f-4357-a6c8-4c504bf26842 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:25.397372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e55f14-921f-4357-a6c8-4c504bf26842 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:25.397655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e55f14-921f-4357-a6c8-4c504bf26842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:25.466047 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:25.471618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-12e55f14-921f-4357-a6c8-4c504bf26842 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:25.479638 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-12e55f14-921f-4357-a6c8-4c504bf26842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:25.479940 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d732b-0e40-4629-b60b-b5a55bb44f51 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:25.654588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 930/1876] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:25.660508 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d27fff-b497-428b-a2db-5808894c1d00 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:25.660798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d27fff-b497-428b-a2db-5808894c1d00 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:25.661081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d27fff-b497-428b-a2db-5808894c1d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:25.674796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808f773-245c-48cc-8ccc-3df47f0df4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:25.675985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808f773-245c-48cc-8ccc-3df47f0df4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:25.676628 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808f773-245c-48cc-8ccc-3df47f0df4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:25.677101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808f773-245c-48cc-8ccc-3df47f0df4cf admin admin] PATH_INFO: `/v3/users/c106703e498b404bae9f242482f23589` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:25.680722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6808f773-245c-48cc-8ccc-3df47f0df4cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c106703e498b404bae9f242482f23589)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:25.682969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6808f773-245c-48cc-8ccc-3df47f0df4cf admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:25.685477 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-91d27fff-b497-428b-a2db-5808894c1d00 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:25.686751 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-91d27fff-b497-428b-a2db-5808894c1d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:25.699714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 931/1877] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:14:25.703873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d27fff-b497-428b-a2db-5808894c1d00 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:25.714330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d27fff-b497-428b-a2db-5808894c1d00 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:25.719710 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d29ee-9124-41a2-b392-863d54ac99af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:25.720411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d29ee-9124-41a2-b392-863d54ac99af admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:25.720678 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d29ee-9124-41a2-b392-863d54ac99af admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:25.720959 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d29ee-9124-41a2-b392-863d54ac99af admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:25.721722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d29ee-9124-41a2-b392-863d54ac99af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:14:25.722040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d29ee-9124-41a2-b392-863d54ac99af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:25.723857 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d29ee-9124-41a2-b392-863d54ac99af admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:25.736499 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 932/1878] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 15:14:25.742371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d29ee-9124-41a2-b392-863d54ac99af admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:25.754155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d29ee-9124-41a2-b392-863d54ac99af admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:25.764132 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27910032-95be-457d-86d6-716d3bc3043b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:25.765553 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27910032-95be-457d-86d6-716d3bc3043b admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:25.765849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27910032-95be-457d-86d6-716d3bc3043b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:25.766127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27910032-95be-457d-86d6-716d3bc3043b admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:25.766938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27910032-95be-457d-86d6-716d3bc3043b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:25.769855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27910032-95be-457d-86d6-716d3bc3043b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:25.904613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 933/1879] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:25 2022] POST /identity/v3/projects => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:14:25.907147 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27910032-95be-457d-86d6-716d3bc3043b admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:25.940875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6808f773-245c-48cc-8ccc-3df47f0df4cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c106703e498b404bae9f242482f23589'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:25.955942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6808f773-245c-48cc-8ccc-3df47f0df4cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c106703e498b404bae9f242482f23589'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:25.955942 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6808f773-245c-48cc-8ccc-3df47f0df4cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c106703e498b404bae9f242482f23589'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:25.969222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6808f773-245c-48cc-8ccc-3df47f0df4cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c106703e498b404bae9f242482f23589'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:25.992080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27910032-95be-457d-86d6-716d3bc3043b admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:25.997298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0fc96-c3b5-458f-b756-940894a1c8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:25.997989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0fc96-c3b5-458f-b756-940894a1c8c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:25.998282 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0fc96-c3b5-458f-b756-940894a1c8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:25.998589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0fc96-c3b5-458f-b756-940894a1c8c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:25.999303 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0fc96-c3b5-458f-b756-940894a1c8c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:26.001288 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0fc96-c3b5-458f-b756-940894a1c8c5 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:26.066181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6808f773-245c-48cc-8ccc-3df47f0df4cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c106703e498b404bae9f242482f23589'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:26.075084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 947/1880] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:25 2022] DELETE /identity/v3/users/c106703e498b404bae9f242482f23589 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:26.102836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 934/1881] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:25 2022] POST /identity/v3/users => generated 284 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:14:26.108052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808f773-245c-48cc-8ccc-3df47f0df4cf admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:26.109667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-6808f773-245c-48cc-8ccc-3df47f0df4cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:26.193671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808f773-245c-48cc-8ccc-3df47f0df4cf admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:26.199733 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5ad7b-36d9-43bf-bd27-f5d6273e231a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:26.200571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5ad7b-36d9-43bf-bd27-f5d6273e231a admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:26.200972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5ad7b-36d9-43bf-bd27-f5d6273e231a admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:26.201160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5ad7b-36d9-43bf-bd27-f5d6273e231a admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:26.201978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5ad7b-36d9-43bf-bd27-f5d6273e231a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:26.203668 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5ad7b-36d9-43bf-bd27-f5d6273e231a admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:26.215076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 948/1882] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:26.220608 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0fc96-c3b5-458f-b756-940894a1c8c5 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:26.231894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0fc96-c3b5-458f-b756-940894a1c8c5 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:26.273475 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cde42-9367-4b3e-bfee-72cec14893a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:26.275733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cde42-9367-4b3e-bfee-72cec14893a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:26.276637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cde42-9367-4b3e-bfee-72cec14893a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:26.278162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cde42-9367-4b3e-bfee-72cec14893a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:26.280021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432cde42-9367-4b3e-bfee-72cec14893a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:26.283860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432cde42-9367-4b3e-bfee-72cec14893a1 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:26.303022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 935/1883] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:26.308437 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5ad7b-36d9-43bf-bd27-f5d6273e231a admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:26.320601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5ad7b-36d9-43bf-bd27-f5d6273e231a admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:26.327481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9fd34-b44d-4eef-818e-d28a79efffc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:26.328361 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9fd34-b44d-4eef-818e-d28a79efffc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:26.329859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9fd34-b44d-4eef-818e-d28a79efffc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:26.329859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9fd34-b44d-4eef-818e-d28a79efffc3 admin admin] PATH_INFO: `/v3/projects/a2022fa1ba714205ac44310bcd814210/users/41fa4478d6484c71a78d20609617f998/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:26.384814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede9fd34-b44d-4eef-818e-d28a79efffc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2022fa1ba714205ac44310bcd814210, user_id=41fa4478d6484c71a78d20609617f998, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:26.387972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede9fd34-b44d-4eef-818e-d28a79efffc3 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:26.440728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:26.545061 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 949/1884] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:26 2022] PUT /identity/v3/projects/a2022fa1ba714205ac44310bcd814210/users/41fa4478d6484c71a78d20609617f998/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:26.566994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19630674-ced2-4d28-9999-1033ba6624d3 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:26.567373 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19630674-ced2-4d28-9999-1033ba6624d3 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:26.569457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19630674-ced2-4d28-9999-1033ba6624d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:26.605245 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-19630674-ced2-4d28-9999-1033ba6624d3 None None] MFA Rules not processed for user `41fa4478d6484c71a78d20609617f998`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:26.606978 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-19630674-ced2-4d28-9999-1033ba6624d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:26.722004 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 936/1885] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:26.726953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9fd34-b44d-4eef-818e-d28a79efffc3 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:26.731964 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19630674-ced2-4d28-9999-1033ba6624d3 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:26.734384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-19630674-ced2-4d28-9999-1033ba6624d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:26.809683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9fd34-b44d-4eef-818e-d28a79efffc3 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:26.830494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8718779-0222-4fb1-be14-3e8e181473ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:26.831389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8718779-0222-4fb1-be14-3e8e181473ac admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:26.831700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8718779-0222-4fb1-be14-3e8e181473ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:26.831979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8718779-0222-4fb1-be14-3e8e181473ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:26.832910 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8718779-0222-4fb1-be14-3e8e181473ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:26.835314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8718779-0222-4fb1-be14-3e8e181473ac admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:26.846715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19630674-ced2-4d28-9999-1033ba6624d3 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:26.867110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a713d-57bc-4ef6-8c5c-9aefc955177d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:26.868698 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a713d-57bc-4ef6-8c5c-9aefc955177d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:26.868698 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a713d-57bc-4ef6-8c5c-9aefc955177d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:26.874375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a713d-57bc-4ef6-8c5c-9aefc955177d admin admin] PATH_INFO: `/v3/projects/16bc9b6f03d442e9a6e0c0d8a6712a9f` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:26.874375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a713d-57bc-4ef6-8c5c-9aefc955177d admin admin] RBAC: Authorizing `identity:delete_project(project_id=16bc9b6f03d442e9a6e0c0d8a6712a9f)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:26.874375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a713d-57bc-4ef6-8c5c-9aefc955177d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:26.969277 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 950/1886] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:26 2022] POST /identity/v3/projects => generated 369 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:26.970947 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8718779-0222-4fb1-be14-3e8e181473ac admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:27.048216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a11a713d-57bc-4ef6-8c5c-9aefc955177d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16bc9b6f03d442e9a6e0c0d8a6712a9f', 'request_id': 'req-a11a713d-57bc-4ef6-8c5c-9aefc955177d'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:27.082996 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a11a713d-57bc-4ef6-8c5c-9aefc955177d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16bc9b6f03d442e9a6e0c0d8a6712a9f', 'request_id': 'req-a11a713d-57bc-4ef6-8c5c-9aefc955177d'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:27.090726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8718779-0222-4fb1-be14-3e8e181473ac admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:27.102697 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a11a713d-57bc-4ef6-8c5c-9aefc955177d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16bc9b6f03d442e9a6e0c0d8a6712a9f', 'request_id': 'req-a11a713d-57bc-4ef6-8c5c-9aefc955177d'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:27.190879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b194f-f683-471b-b339-8e05e62fb699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:27.192280 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b194f-f683-471b-b339-8e05e62fb699 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:27.193085 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b194f-f683-471b-b339-8e05e62fb699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:27.193601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b194f-f683-471b-b339-8e05e62fb699 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:27.195104 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b194f-f683-471b-b339-8e05e62fb699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:27.199656 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b194f-f683-471b-b339-8e05e62fb699 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:27.204234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a11a713d-57bc-4ef6-8c5c-9aefc955177d admin admin] The token cache is being invalidate because project 16bc9b6f03d442e9a6e0c0d8a6712a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:27.213570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a11a713d-57bc-4ef6-8c5c-9aefc955177d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:27.213570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a11a713d-57bc-4ef6-8c5c-9aefc955177d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:27.213570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 937/1887] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:26 2022] DELETE /identity/v3/projects/16bc9b6f03d442e9a6e0c0d8a6712a9f => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:27.216143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dac833-d55f-4328-af79-556134b1ba49 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:27.216520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dac833-d55f-4328-af79-556134b1ba49 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:27.216879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dac833-d55f-4328-af79-556134b1ba49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:27.242258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-33dac833-d55f-4328-af79-556134b1ba49 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:27.243620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-33dac833-d55f-4328-af79-556134b1ba49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:27.263119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 938/1888] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:27.265697 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaa80b-6242-4360-8ce4-69267b551681 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:27.266175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaa80b-6242-4360-8ce4-69267b551681 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:27.266667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaa80b-6242-4360-8ce4-69267b551681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:27.291938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-9deaa80b-6242-4360-8ce4-69267b551681 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:27.295485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9deaa80b-6242-4360-8ce4-69267b551681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:27.301918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 951/1889] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:26 2022] POST /identity/v3/users => generated 291 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:27.308703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b194f-f683-471b-b339-8e05e62fb699 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:27.311776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ca4b194f-f683-471b-b339-8e05e62fb699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:27.311994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 939/1890] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:27.318338 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deaa80b-6242-4360-8ce4-69267b551681 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:27.329918 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deaa80b-6242-4360-8ce4-69267b551681 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:27.337985 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8b6d0-16cf-4210-80c7-9defb4ca4da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:27.338957 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8b6d0-16cf-4210-80c7-9defb4ca4da5 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:27.339194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b194f-f683-471b-b339-8e05e62fb699 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:27.339335 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8b6d0-16cf-4210-80c7-9defb4ca4da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:27.339590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8b6d0-16cf-4210-80c7-9defb4ca4da5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:27.340388 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8b6d0-16cf-4210-80c7-9defb4ca4da5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:14:27.340706 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8b6d0-16cf-4210-80c7-9defb4ca4da5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:27.342378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8b6d0-16cf-4210-80c7-9defb4ca4da5 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:27.348719 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403a57b-61b7-4c0b-a61b-2547bcc36106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:27.350511 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403a57b-61b7-4c0b-a61b-2547bcc36106 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:27.350511 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403a57b-61b7-4c0b-a61b-2547bcc36106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:27.350889 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403a57b-61b7-4c0b-a61b-2547bcc36106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:27.351829 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403a57b-61b7-4c0b-a61b-2547bcc36106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:27.354056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403a57b-61b7-4c0b-a61b-2547bcc36106 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:27.355754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:27.366004 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 940/1891] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 15:14:27.366563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 952/1892] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:14:27.370474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8b6d0-16cf-4210-80c7-9defb4ca4da5 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:27.370967 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403a57b-61b7-4c0b-a61b-2547bcc36106 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:27.384148 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403a57b-61b7-4c0b-a61b-2547bcc36106 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:27.391165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8b6d0-16cf-4210-80c7-9defb4ca4da5 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:27.394239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb97ebe-1c51-4fa7-b6ff-c9c612be2008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:27.395490 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb97ebe-1c51-4fa7-b6ff-c9c612be2008 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:27.396479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb97ebe-1c51-4fa7-b6ff-c9c612be2008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:27.397544 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb97ebe-1c51-4fa7-b6ff-c9c612be2008 admin admin] PATH_INFO: `/v3/projects/6ea8012c74e144f8b46cfa773349a348/users/6a26a1f816624990b64d95122dbe1880/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:27.402792 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141191c-82c2-42ba-b910-0ef4e5917140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:27.403591 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141191c-82c2-42ba-b910-0ef4e5917140 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:27.404018 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141191c-82c2-42ba-b910-0ef4e5917140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:27.416687 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141191c-82c2-42ba-b910-0ef4e5917140 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:27.423122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141191c-82c2-42ba-b910-0ef4e5917140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:27.423122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141191c-82c2-42ba-b910-0ef4e5917140 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:27.434365 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb97ebe-1c51-4fa7-b6ff-c9c612be2008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ea8012c74e144f8b46cfa773349a348, user_id=6a26a1f816624990b64d95122dbe1880, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:27.437575 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb97ebe-1c51-4fa7-b6ff-c9c612be2008 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:27.614829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 941/1894] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:27 2022] POST /identity/v3/projects => generated 373 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:14:27.615597 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 953/1894] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:27 2022] PUT /identity/v3/projects/6ea8012c74e144f8b46cfa773349a348/users/6a26a1f816624990b64d95122dbe1880/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:27.617323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141191c-82c2-42ba-b910-0ef4e5917140 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:27.617655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb97ebe-1c51-4fa7-b6ff-c9c612be2008 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:27.746792 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb97ebe-1c51-4fa7-b6ff-c9c612be2008 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:27.753691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ec6ee-9845-4c07-848d-d38b8690810d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:27.754722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ec6ee-9845-4c07-848d-d38b8690810d admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:27.755176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ec6ee-9845-4c07-848d-d38b8690810d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:27.755600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ec6ee-9845-4c07-848d-d38b8690810d admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:27.756397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ec6ee-9845-4c07-848d-d38b8690810d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:27.758586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ec6ee-9845-4c07-848d-d38b8690810d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:27.759846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141191c-82c2-42ba-b910-0ef4e5917140 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:27.768923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e6338-e418-4db8-93b5-efc01ed869ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:27.770400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e6338-e418-4db8-93b5-efc01ed869ea admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:27.772539 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e6338-e418-4db8-93b5-efc01ed869ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:27.772996 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e6338-e418-4db8-93b5-efc01ed869ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:27.776336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e6338-e418-4db8-93b5-efc01ed869ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:27.778049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e6338-e418-4db8-93b5-efc01ed869ea admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:27.805714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 942/1895] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:14:27.819907 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e6338-e418-4db8-93b5-efc01ed869ea admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:27.857811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 954/1896] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:27 2022] POST /identity/v3/users => generated 287 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:27.860137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ec6ee-9845-4c07-848d-d38b8690810d admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:27.865919 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e6338-e418-4db8-93b5-efc01ed869ea admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:27.898504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ec6ee-9845-4c07-848d-d38b8690810d admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:27.905934 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abe11a-c2b8-4ef0-8ce7-deb7f7c71e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:27.907054 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abe11a-c2b8-4ef0-8ce7-deb7f7c71e5e admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:27.911788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b9d9d-0f69-4309-8905-cb572010fb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:27.913424 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b9d9d-0f69-4309-8905-cb572010fb56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:27.913948 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b9d9d-0f69-4309-8905-cb572010fb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:27.920406 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b9d9d-0f69-4309-8905-cb572010fb56 admin admin] PATH_INFO: `/v3/projects/6ea8012c74e144f8b46cfa773349a348/users/6a26a1f816624990b64d95122dbe1880/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:27.921243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abe11a-c2b8-4ef0-8ce7-deb7f7c71e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:27.921727 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abe11a-c2b8-4ef0-8ce7-deb7f7c71e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:27.922396 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abe11a-c2b8-4ef0-8ce7-deb7f7c71e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:27.923116 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b9d9d-0f69-4309-8905-cb572010fb56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ea8012c74e144f8b46cfa773349a348, user_id=6a26a1f816624990b64d95122dbe1880, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:27.929776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b9d9d-0f69-4309-8905-cb572010fb56 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:27.931013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abe11a-c2b8-4ef0-8ce7-deb7f7c71e5e admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:27.964825 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 943/1897] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:27 2022] PUT /identity/v3/projects/6ea8012c74e144f8b46cfa773349a348/users/6a26a1f816624990b64d95122dbe1880/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:27.967091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 955/1898] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:27.971011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c209b0-67d5-4c42-8cb1-427e6ebe5cac None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:27.971646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c209b0-67d5-4c42-8cb1-427e6ebe5cac None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:27.972462 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c209b0-67d5-4c42-8cb1-427e6ebe5cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:27.978570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abe11a-c2b8-4ef0-8ce7-deb7f7c71e5e admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:27.983316 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:28.017653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abe11a-c2b8-4ef0-8ce7-deb7f7c71e5e admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:28.031483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-02c209b0-67d5-4c42-8cb1-427e6ebe5cac None None] MFA Rules not processed for user `6a26a1f816624990b64d95122dbe1880`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:28.033442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-02c209b0-67d5-4c42-8cb1-427e6ebe5cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:28.115130 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d3c59-34d0-43e7-9ffd-0d20e7fcdffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:28.127344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d3c59-34d0-43e7-9ffd-0d20e7fcdffc admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:28.128093 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d3c59-34d0-43e7-9ffd-0d20e7fcdffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:28.128668 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d3c59-34d0-43e7-9ffd-0d20e7fcdffc admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:28.135861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d3c59-34d0-43e7-9ffd-0d20e7fcdffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:28.137870 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d3c59-34d0-43e7-9ffd-0d20e7fcdffc admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:28.171796 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 944/1899] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:27 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:28.172242 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 956/1900] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:28.183457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c209b0-67d5-4c42-8cb1-427e6ebe5cac None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:28.185075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d3c59-34d0-43e7-9ffd-0d20e7fcdffc admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:28.186826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-763d3c59-34d0-43e7-9ffd-0d20e7fcdffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:28.196378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c209b0-67d5-4c42-8cb1-427e6ebe5cac None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:28.202822 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e19d52-ec96-4f9c-a2ec-1f1f278cfe0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:28.203927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e19d52-ec96-4f9c-a2ec-1f1f278cfe0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:28.204898 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e19d52-ec96-4f9c-a2ec-1f1f278cfe0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:28.205481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e19d52-ec96-4f9c-a2ec-1f1f278cfe0f admin admin] PATH_INFO: `/v3/projects/4cdfab02f37c4b678a1628c4a657f1ae/users/4986898a0bff4f339959e3ffd9f30945/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:28.226722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e19d52-ec96-4f9c-a2ec-1f1f278cfe0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cdfab02f37c4b678a1628c4a657f1ae, user_id=4986898a0bff4f339959e3ffd9f30945, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:28.232912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e19d52-ec96-4f9c-a2ec-1f1f278cfe0f admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:28.292046 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d3c59-34d0-43e7-9ffd-0d20e7fcdffc admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:28.295501 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:28.296696 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbc4c4-f409-433f-aab9-3b5d0fa69736 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:28.297446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbc4c4-f409-433f-aab9-3b5d0fa69736 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:28.297815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbc4c4-f409-433f-aab9-3b5d0fa69736 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:28.298179 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbc4c4-f409-433f-aab9-3b5d0fa69736 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:28.309980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbc4c4-f409-433f-aab9-3b5d0fa69736 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:28.311681 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbc4c4-f409-433f-aab9-3b5d0fa69736 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:28.323989 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 957/1901] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:28 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:28.386598 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 945/1902] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:28 2022] PUT /identity/v3/projects/4cdfab02f37c4b678a1628c4a657f1ae/users/4986898a0bff4f339959e3ffd9f30945/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:14:28.397930 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe8547-7902-4caf-91fa-d3fb64716522 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:28.398210 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe8547-7902-4caf-91fa-d3fb64716522 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:28.398493 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe8547-7902-4caf-91fa-d3fb64716522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:28.429765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-96fe8547-7902-4caf-91fa-d3fb64716522 None None] MFA Rules not processed for user `4986898a0bff4f339959e3ffd9f30945`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:28.431103 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-96fe8547-7902-4caf-91fa-d3fb64716522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:28.482229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e19d52-ec96-4f9c-a2ec-1f1f278cfe0f admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:28.483945 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-90e19d52-ec96-4f9c-a2ec-1f1f278cfe0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:28.527371 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 958/1903] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:14:28.545680 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe8547-7902-4caf-91fa-d3fb64716522 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:28.553893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e19d52-ec96-4f9c-a2ec-1f1f278cfe0f admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:28.558695 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0167420-c08c-422a-8cbd-18f2b7c5d82b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:28.558961 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0167420-c08c-422a-8cbd-18f2b7c5d82b service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:28.559380 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0167420-c08c-422a-8cbd-18f2b7c5d82b service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:28.559761 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0167420-c08c-422a-8cbd-18f2b7c5d82b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:28.562659 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c0167420-c08c-422a-8cbd-18f2b7c5d82b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:28.569009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe8547-7902-4caf-91fa-d3fb64716522 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:28.573064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d8a09-f50a-42a3-8f3a-1ea43795a977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:28.573804 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d8a09-f50a-42a3-8f3a-1ea43795a977 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:28.574104 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d8a09-f50a-42a3-8f3a-1ea43795a977 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:28.574390 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d8a09-f50a-42a3-8f3a-1ea43795a977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:28.585039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d8a09-f50a-42a3-8f3a-1ea43795a977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:28.586923 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d8a09-f50a-42a3-8f3a-1ea43795a977 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:28.602863 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 959/1904] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:28.633555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d8a09-f50a-42a3-8f3a-1ea43795a977 service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:28.670612 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0167420-c08c-422a-8cbd-18f2b7c5d82b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:28.671298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0167420-c08c-422a-8cbd-18f2b7c5d82b service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:28.687153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 946/1905] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:28.735936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d8a09-f50a-42a3-8f3a-1ea43795a977 service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:28.742798 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef35795-6119-42d9-a0c9-804aaf7b9d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:28.743697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef35795-6119-42d9-a0c9-804aaf7b9d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:28.743984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef35795-6119-42d9-a0c9-804aaf7b9d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:28.744290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef35795-6119-42d9-a0c9-804aaf7b9d48 admin admin] PATH_INFO: `/v3/users/4986898a0bff4f339959e3ffd9f30945` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:28.745870 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef35795-6119-42d9-a0c9-804aaf7b9d48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4986898a0bff4f339959e3ffd9f30945)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:28.748570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef35795-6119-42d9-a0c9-804aaf7b9d48 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:29.009979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1ef35795-6119-42d9-a0c9-804aaf7b9d48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4986898a0bff4f339959e3ffd9f30945'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:29.018161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1ef35795-6119-42d9-a0c9-804aaf7b9d48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4986898a0bff4f339959e3ffd9f30945'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:29.019193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1ef35795-6119-42d9-a0c9-804aaf7b9d48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4986898a0bff4f339959e3ffd9f30945'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:29.036979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1ef35795-6119-42d9-a0c9-804aaf7b9d48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4986898a0bff4f339959e3ffd9f30945'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:29.192842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1ef35795-6119-42d9-a0c9-804aaf7b9d48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4986898a0bff4f339959e3ffd9f30945'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:29.197148 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 960/1906] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:28 2022] DELETE /identity/v3/users/4986898a0bff4f339959e3ffd9f30945 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:14:29.202228 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b119f-30aa-4718-bee1-99967732d994 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:29.202858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b119f-30aa-4718-bee1-99967732d994 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:29.202858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b119f-30aa-4718-bee1-99967732d994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:29.235065 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-530b119f-30aa-4718-bee1-99967732d994 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:29.236789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-530b119f-30aa-4718-bee1-99967732d994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:29.320202 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 947/1907] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:29.339777 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef35795-6119-42d9-a0c9-804aaf7b9d48 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:29.341404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1ef35795-6119-42d9-a0c9-804aaf7b9d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:29.394872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b119f-30aa-4718-bee1-99967732d994 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:29.397159 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-530b119f-30aa-4718-bee1-99967732d994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:29.453595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b119f-30aa-4718-bee1-99967732d994 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:29.460759 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef35795-6119-42d9-a0c9-804aaf7b9d48 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:29.462528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861d0b0-d898-44a3-9ff7-289b99ee5452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:29.463496 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861d0b0-d898-44a3-9ff7-289b99ee5452 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:29.463860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861d0b0-d898-44a3-9ff7-289b99ee5452 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:29.464338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861d0b0-d898-44a3-9ff7-289b99ee5452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:29.472987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ee36c-d476-4649-8265-353597b89c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:29.475175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ee36c-d476-4649-8265-353597b89c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:29.475493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ee36c-d476-4649-8265-353597b89c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:29.475783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ee36c-d476-4649-8265-353597b89c11 admin admin] PATH_INFO: `/v3/users/e70279b0cd7242058d70d2e9dec4666a` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:29.483168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ee36c-d476-4649-8265-353597b89c11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e70279b0cd7242058d70d2e9dec4666a)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:29.485644 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ee36c-d476-4649-8265-353597b89c11 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:29.500432 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861d0b0-d898-44a3-9ff7-289b99ee5452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:29.512620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861d0b0-d898-44a3-9ff7-289b99ee5452 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:29.531087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 961/1908] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:29.768230 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b66ee36c-d476-4649-8265-353597b89c11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e70279b0cd7242058d70d2e9dec4666a'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:29.771067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b66ee36c-d476-4649-8265-353597b89c11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e70279b0cd7242058d70d2e9dec4666a'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:29.772209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b66ee36c-d476-4649-8265-353597b89c11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e70279b0cd7242058d70d2e9dec4666a'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:29.867771 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b66ee36c-d476-4649-8265-353597b89c11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e70279b0cd7242058d70d2e9dec4666a'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:29.897166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b66ee36c-d476-4649-8265-353597b89c11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e70279b0cd7242058d70d2e9dec4666a'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:29.900890 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:29.914984 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 948/1909] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:29 2022] DELETE /identity/v3/users/e70279b0cd7242058d70d2e9dec4666a => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:29.920186 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413be5d-ade2-43a3-9e32-3f9da215f1ca None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:29.920536 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413be5d-ade2-43a3-9e32-3f9da215f1ca None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:29.920825 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413be5d-ade2-43a3-9e32-3f9da215f1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:29.946736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-7413be5d-ade2-43a3-9e32-3f9da215f1ca None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:29.948110 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7413be5d-ade2-43a3-9e32-3f9da215f1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:30.042017 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 962/1910] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:30.091062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:30.108371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ee36c-d476-4649-8265-353597b89c11 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:30.112190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b66ee36c-d476-4649-8265-353597b89c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:30.223329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ee36c-d476-4649-8265-353597b89c11 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:30.228606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5b47e-ee16-4ed0-84b7-567f86afe3ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:30.229287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5b47e-ee16-4ed0-84b7-567f86afe3ab service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:30.229570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5b47e-ee16-4ed0-84b7-567f86afe3ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:30.229847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5b47e-ee16-4ed0-84b7-567f86afe3ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:30.243343 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5b47e-ee16-4ed0-84b7-567f86afe3ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:30.245488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5b47e-ee16-4ed0-84b7-567f86afe3ab service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:30.259593 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 949/1911] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:30.946677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413be5d-ade2-43a3-9e32-3f9da215f1ca None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:30.948791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7413be5d-ade2-43a3-9e32-3f9da215f1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:30.961616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413be5d-ade2-43a3-9e32-3f9da215f1ca None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:30.964978 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:30.966749 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cb579-10e7-4140-9734-755cfa98d60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:30.967910 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cb579-10e7-4140-9734-755cfa98d60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:30.968762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cb579-10e7-4140-9734-755cfa98d60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:30.969234 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cb579-10e7-4140-9734-755cfa98d60c admin admin] PATH_INFO: `/v3/projects/4cdfab02f37c4b678a1628c4a657f1ae` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:30.970922 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0cb579-10e7-4140-9734-755cfa98d60c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cdfab02f37c4b678a1628c4a657f1ae)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:30.973210 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0cb579-10e7-4140-9734-755cfa98d60c admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:31.164483 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9f0cb579-10e7-4140-9734-755cfa98d60c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cdfab02f37c4b678a1628c4a657f1ae', 'request_id': 'req-9f0cb579-10e7-4140-9734-755cfa98d60c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:31.175389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9f0cb579-10e7-4140-9734-755cfa98d60c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cdfab02f37c4b678a1628c4a657f1ae', 'request_id': 'req-9f0cb579-10e7-4140-9734-755cfa98d60c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:31.257156 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:31.286940 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9f0cb579-10e7-4140-9734-755cfa98d60c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cdfab02f37c4b678a1628c4a657f1ae', 'request_id': 'req-9f0cb579-10e7-4140-9734-755cfa98d60c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:31.310138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9f0cb579-10e7-4140-9734-755cfa98d60c admin admin] The token cache is being invalidate because project 4cdfab02f37c4b678a1628c4a657f1ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:31.310499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9f0cb579-10e7-4140-9734-755cfa98d60c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:31.313912 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9f0cb579-10e7-4140-9734-755cfa98d60c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:31.318265 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 963/1912] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:30 2022] DELETE /identity/v3/projects/4cdfab02f37c4b678a1628c4a657f1ae => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:31.323843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808137b-ce1d-4ac4-866f-1062f222e1c3 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:31.325370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808137b-ce1d-4ac4-866f-1062f222e1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:31.345459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808137b-ce1d-4ac4-866f-1062f222e1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:31.364307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-8808137b-ce1d-4ac4-866f-1062f222e1c3 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:31.366207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-8808137b-ce1d-4ac4-866f-1062f222e1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:31.497168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 950/1913] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:31.510703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae199b85-0acd-402f-a7e0-7632d98ad921 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:31.510703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae199b85-0acd-402f-a7e0-7632d98ad921 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:31.511248 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae199b85-0acd-402f-a7e0-7632d98ad921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:31.557852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-ae199b85-0acd-402f-a7e0-7632d98ad921 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:31.559307 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ae199b85-0acd-402f-a7e0-7632d98ad921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:31.578037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 964/1914] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:31.581147 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808137b-ce1d-4ac4-866f-1062f222e1c3 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:31.594254 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808137b-ce1d-4ac4-866f-1062f222e1c3 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:31.610656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a2bfe-1f15-435b-b064-b5a79644c763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:31.611653 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a2bfe-1f15-435b-b064-b5a79644c763 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:31.612154 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a2bfe-1f15-435b-b064-b5a79644c763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:31.613861 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a2bfe-1f15-435b-b064-b5a79644c763 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:31.615409 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a2bfe-1f15-435b-b064-b5a79644c763 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:14:31.618201 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a2bfe-1f15-435b-b064-b5a79644c763 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:31.620068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a2bfe-1f15-435b-b064-b5a79644c763 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:31.654184 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 951/1915] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:14:31.658709 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae199b85-0acd-402f-a7e0-7632d98ad921 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:31.673123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae199b85-0acd-402f-a7e0-7632d98ad921 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:31.680687 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f442498-f9cb-4924-9a3b-e7d82df69ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:31.681503 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f442498-f9cb-4924-9a3b-e7d82df69ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:31.681866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f442498-f9cb-4924-9a3b-e7d82df69ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:31.682223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f442498-f9cb-4924-9a3b-e7d82df69ea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:31.683427 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f442498-f9cb-4924-9a3b-e7d82df69ea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:31.685532 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f442498-f9cb-4924-9a3b-e7d82df69ea2 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:31.806352 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:31.847466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 965/1916] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:31 2022] POST /identity/v3/projects => generated 375 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:31.860358 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a2bfe-1f15-435b-b064-b5a79644c763 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:31.861738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f442498-f9cb-4924-9a3b-e7d82df69ea2 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:31.863571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2c2a2bfe-1f15-435b-b064-b5a79644c763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:31.890572 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a2bfe-1f15-435b-b064-b5a79644c763 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:32.001623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f442498-f9cb-4924-9a3b-e7d82df69ea2 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:32.007955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946b436-5c39-40a7-b917-c0fd97d7713e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:32.008905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946b436-5c39-40a7-b917-c0fd97d7713e admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:32.009407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946b436-5c39-40a7-b917-c0fd97d7713e admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:32.009690 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946b436-5c39-40a7-b917-c0fd97d7713e admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:32.010584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946b436-5c39-40a7-b917-c0fd97d7713e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:32.013102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946b436-5c39-40a7-b917-c0fd97d7713e admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:32.035016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575eba2-e061-4e4f-9284-aff8e0ae8266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:32.035995 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575eba2-e061-4e4f-9284-aff8e0ae8266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:32.036279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575eba2-e061-4e4f-9284-aff8e0ae8266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:32.036651 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575eba2-e061-4e4f-9284-aff8e0ae8266 admin admin] PATH_INFO: `/v3/projects/c89a4abfc10747478072af1a662da4b6` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:32.038096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575eba2-e061-4e4f-9284-aff8e0ae8266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c89a4abfc10747478072af1a662da4b6)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:32.040245 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575eba2-e061-4e4f-9284-aff8e0ae8266 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:32.209296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 966/1917] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:31 2022] POST /identity/v3/users => generated 288 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:32.212767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946b436-5c39-40a7-b917-c0fd97d7713e admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:32.238165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946b436-5c39-40a7-b917-c0fd97d7713e admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:32.269791 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:32.274273 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:32.276779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9575eba2-e061-4e4f-9284-aff8e0ae8266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c89a4abfc10747478072af1a662da4b6', 'request_id': 'req-9575eba2-e061-4e4f-9284-aff8e0ae8266'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:32.288748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9575eba2-e061-4e4f-9284-aff8e0ae8266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c89a4abfc10747478072af1a662da4b6', 'request_id': 'req-9575eba2-e061-4e4f-9284-aff8e0ae8266'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:32.319753 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9575eba2-e061-4e4f-9284-aff8e0ae8266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c89a4abfc10747478072af1a662da4b6', 'request_id': 'req-9575eba2-e061-4e4f-9284-aff8e0ae8266'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:32.368654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf368507-10b3-45a3-a793-1af1003ede84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:32.369854 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf368507-10b3-45a3-a793-1af1003ede84 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:32.370145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf368507-10b3-45a3-a793-1af1003ede84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:32.370476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf368507-10b3-45a3-a793-1af1003ede84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:32.374325 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf368507-10b3-45a3-a793-1af1003ede84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:32.374325 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf368507-10b3-45a3-a793-1af1003ede84 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:32.398714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 967/1918] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:32.405477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf368507-10b3-45a3-a793-1af1003ede84 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:32.423284 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf368507-10b3-45a3-a793-1af1003ede84 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:32.433746 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23869cc9-7443-4eae-b0bd-328ad07d61a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:32.434474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23869cc9-7443-4eae-b0bd-328ad07d61a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:32.434952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23869cc9-7443-4eae-b0bd-328ad07d61a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:32.436169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23869cc9-7443-4eae-b0bd-328ad07d61a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:32.437053 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23869cc9-7443-4eae-b0bd-328ad07d61a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:32.439631 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23869cc9-7443-4eae-b0bd-328ad07d61a6 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:32.457113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 968/1919] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:32.463937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23869cc9-7443-4eae-b0bd-328ad07d61a6 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:32.479942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9575eba2-e061-4e4f-9284-aff8e0ae8266 admin admin] The token cache is being invalidate because project c89a4abfc10747478072af1a662da4b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:32.480660 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9575eba2-e061-4e4f-9284-aff8e0ae8266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:32.486615 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9575eba2-e061-4e4f-9284-aff8e0ae8266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:32.486615 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 952/1920] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:31 2022] DELETE /identity/v3/projects/c89a4abfc10747478072af1a662da4b6 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:32.491014 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575eba2-e061-4e4f-9284-aff8e0ae8266 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:32.492177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23869cc9-7443-4eae-b0bd-328ad07d61a6 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:32.494446 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9575eba2-e061-4e4f-9284-aff8e0ae8266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:32.497547 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3e6eb-c53e-4fa0-9e49-923a09aa30af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:32.498472 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3e6eb-c53e-4fa0-9e49-923a09aa30af admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:32.498868 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3e6eb-c53e-4fa0-9e49-923a09aa30af admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:32.499156 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3e6eb-c53e-4fa0-9e49-923a09aa30af admin admin] PATH_INFO: `/v3/projects/402857c697be47abb330793e851f6f45/users/41e480fc338449d2a51e13cc36ebe729/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:32.529096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3e6eb-c53e-4fa0-9e49-923a09aa30af admin admin] RBAC: Authorizing `identity:create_grant(project_id=402857c697be47abb330793e851f6f45, user_id=41e480fc338449d2a51e13cc36ebe729, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:32.532007 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3e6eb-c53e-4fa0-9e49-923a09aa30af admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:32.540650 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575eba2-e061-4e4f-9284-aff8e0ae8266 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:32.560472 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c969325-8fe0-4f5e-a870-f83f9ada0b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:32.562422 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c969325-8fe0-4f5e-a870-f83f9ada0b7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:32.563034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c969325-8fe0-4f5e-a870-f83f9ada0b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:32.563693 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c969325-8fe0-4f5e-a870-f83f9ada0b7a admin admin] PATH_INFO: `/v3/users/506f816cd9624e51bb55872bbf9747ad` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:32.565454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c969325-8fe0-4f5e-a870-f83f9ada0b7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=506f816cd9624e51bb55872bbf9747ad)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:32.581993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c969325-8fe0-4f5e-a870-f83f9ada0b7a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:32.644999 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 969/1921] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:32 2022] PUT /identity/v3/projects/402857c697be47abb330793e851f6f45/users/41e480fc338449d2a51e13cc36ebe729/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:32.657118 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9752b1b-b1e5-4ab7-a9eb-f49eba5c7c09 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:32.657562 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9752b1b-b1e5-4ab7-a9eb-f49eba5c7c09 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:32.657847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9752b1b-b1e5-4ab7-a9eb-f49eba5c7c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:32.709191 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-a9752b1b-b1e5-4ab7-a9eb-f49eba5c7c09 None None] MFA Rules not processed for user `41e480fc338449d2a51e13cc36ebe729`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:32.711463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a9752b1b-b1e5-4ab7-a9eb-f49eba5c7c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:32.807918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 970/1922] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:14:32.822115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9752b1b-b1e5-4ab7-a9eb-f49eba5c7c09 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:32.823628 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a9752b1b-b1e5-4ab7-a9eb-f49eba5c7c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:32.881895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-7c969325-8fe0-4f5e-a870-f83f9ada0b7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '506f816cd9624e51bb55872bbf9747ad'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:32.883076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-7c969325-8fe0-4f5e-a870-f83f9ada0b7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '506f816cd9624e51bb55872bbf9747ad'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:32.883910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-7c969325-8fe0-4f5e-a870-f83f9ada0b7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '506f816cd9624e51bb55872bbf9747ad'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:32.917987 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9752b1b-b1e5-4ab7-a9eb-f49eba5c7c09 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:32.921881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f2538-df05-448b-8a99-c8dfb6815f1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:32.922574 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f2538-df05-448b-8a99-c8dfb6815f1a service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:32.922876 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f2538-df05-448b-8a99-c8dfb6815f1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:32.923160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f2538-df05-448b-8a99-c8dfb6815f1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:32.928588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-846f2538-df05-448b-8a99-c8dfb6815f1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:32.941125 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f2538-df05-448b-8a99-c8dfb6815f1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:32.942835 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f2538-df05-448b-8a99-c8dfb6815f1a service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:32.953922 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 971/1923] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:14:33.013372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-7c969325-8fe0-4f5e-a870-f83f9ada0b7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '506f816cd9624e51bb55872bbf9747ad'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:33.043564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-7c969325-8fe0-4f5e-a870-f83f9ada0b7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '506f816cd9624e51bb55872bbf9747ad'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:33.062112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 953/1924] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:32 2022] DELETE /identity/v3/users/506f816cd9624e51bb55872bbf9747ad => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:14:33.485206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:34.152428 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f2538-df05-448b-8a99-c8dfb6815f1a service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:34.156311 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-846f2538-df05-448b-8a99-c8dfb6815f1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:34.375421 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f2538-df05-448b-8a99-c8dfb6815f1a service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:34.449706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c694c9-fb0b-4a94-94b8-3b054268de8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:34.450958 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c694c9-fb0b-4a94-94b8-3b054268de8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:34.451416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c694c9-fb0b-4a94-94b8-3b054268de8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:34.451865 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c694c9-fb0b-4a94-94b8-3b054268de8a admin admin] PATH_INFO: `/v3/projects/cee52315be764f5d8741086091a28aed` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:34.453450 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c694c9-fb0b-4a94-94b8-3b054268de8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee52315be764f5d8741086091a28aed)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:34.455627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c694c9-fb0b-4a94-94b8-3b054268de8a admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:34.593342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-51c694c9-fb0b-4a94-94b8-3b054268de8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee52315be764f5d8741086091a28aed', 'request_id': 'req-51c694c9-fb0b-4a94-94b8-3b054268de8a'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:34.602456 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-51c694c9-fb0b-4a94-94b8-3b054268de8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee52315be764f5d8741086091a28aed', 'request_id': 'req-51c694c9-fb0b-4a94-94b8-3b054268de8a'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:34.749524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-51c694c9-fb0b-4a94-94b8-3b054268de8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee52315be764f5d8741086091a28aed', 'request_id': 'req-51c694c9-fb0b-4a94-94b8-3b054268de8a'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:34.782644 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-51c694c9-fb0b-4a94-94b8-3b054268de8a admin admin] The token cache is being invalidate because project cee52315be764f5d8741086091a28aed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:34.783092 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-51c694c9-fb0b-4a94-94b8-3b054268de8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:34.785580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-51c694c9-fb0b-4a94-94b8-3b054268de8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:34.788729 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 972/1925] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:34 2022] DELETE /identity/v3/projects/cee52315be764f5d8741086091a28aed => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:34.793842 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861fb85-8bd6-4c6a-be42-dcdf7cebf3fd None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:34.794460 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861fb85-8bd6-4c6a-be42-dcdf7cebf3fd None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:34.794769 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861fb85-8bd6-4c6a-be42-dcdf7cebf3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:34.832849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:34.838357 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-a861fb85-8bd6-4c6a-be42-dcdf7cebf3fd None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:34.840812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a861fb85-8bd6-4c6a-be42-dcdf7cebf3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:34.923428 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 954/1926] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:34.933625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58062cf8-f8ed-462e-b5e6-a2ced499c1eb None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:34.934295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58062cf8-f8ed-462e-b5e6-a2ced499c1eb None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:34.934661 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58062cf8-f8ed-462e-b5e6-a2ced499c1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:34.958091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-58062cf8-f8ed-462e-b5e6-a2ced499c1eb None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:34.960166 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-58062cf8-f8ed-462e-b5e6-a2ced499c1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:34.969361 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 973/1927] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:14:34.974223 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861fb85-8bd6-4c6a-be42-dcdf7cebf3fd None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:34.985997 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861fb85-8bd6-4c6a-be42-dcdf7cebf3fd None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:34.996530 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6b4f5-d2d1-4956-859f-123cddf00097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:34.997843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6b4f5-d2d1-4956-859f-123cddf00097 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:34.997843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6b4f5-d2d1-4956-859f-123cddf00097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:34.998398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6b4f5-d2d1-4956-859f-123cddf00097 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:34.999722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6b4f5-d2d1-4956-859f-123cddf00097 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:14:35.000022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6b4f5-d2d1-4956-859f-123cddf00097 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:35.001657 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6b4f5-d2d1-4956-859f-123cddf00097 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:35.020891 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 955/1928] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:14:35.030522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58062cf8-f8ed-462e-b5e6-a2ced499c1eb None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:35.040248 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58062cf8-f8ed-462e-b5e6-a2ced499c1eb None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:35.048899 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42913b5-53f7-4d8c-a5df-7c622bec3859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:35.049763 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42913b5-53f7-4d8c-a5df-7c622bec3859 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:35.050209 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42913b5-53f7-4d8c-a5df-7c622bec3859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:35.050760 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42913b5-53f7-4d8c-a5df-7c622bec3859 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:35.051556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42913b5-53f7-4d8c-a5df-7c622bec3859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:35.055477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42913b5-53f7-4d8c-a5df-7c622bec3859 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:35.206286 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 974/1929] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:35 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:35.207802 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6b4f5-d2d1-4956-859f-123cddf00097 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:35.220609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6b4f5-d2d1-4956-859f-123cddf00097 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:35.259957 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:35.295144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b1f0e-7c02-4a50-b33e-638a5c08fcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:35.295845 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b1f0e-7c02-4a50-b33e-638a5c08fcb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:35.296129 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b1f0e-7c02-4a50-b33e-638a5c08fcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:35.296432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b1f0e-7c02-4a50-b33e-638a5c08fcb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:35.297318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b1f0e-7c02-4a50-b33e-638a5c08fcb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:35.299431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b1f0e-7c02-4a50-b33e-638a5c08fcb0 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:35.443131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 956/1930] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:35 2022] POST /identity/v3/users => generated 288 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:14:35.444356 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42913b5-53f7-4d8c-a5df-7c622bec3859 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:35.465549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42913b5-53f7-4d8c-a5df-7c622bec3859 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:35.471893 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d2988-0214-4e63-865c-2e7119c28ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:35.472512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d2988-0214-4e63-865c-2e7119c28ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:35.472801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d2988-0214-4e63-865c-2e7119c28ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:35.473087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d2988-0214-4e63-865c-2e7119c28ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:35.473878 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d2988-0214-4e63-865c-2e7119c28ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:35.476703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d2988-0214-4e63-865c-2e7119c28ab2 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:35.491636 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 975/1931] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:35.496090 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b1f0e-7c02-4a50-b33e-638a5c08fcb0 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:35.523437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b1f0e-7c02-4a50-b33e-638a5c08fcb0 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:35.529455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac911920-b6ad-438d-898a-2b8354c74b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:35.534235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac911920-b6ad-438d-898a-2b8354c74b50 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:35.534235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac911920-b6ad-438d-898a-2b8354c74b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:35.534235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac911920-b6ad-438d-898a-2b8354c74b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:35.534235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac911920-b6ad-438d-898a-2b8354c74b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:35.534735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac911920-b6ad-438d-898a-2b8354c74b50 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:35.547476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 957/1932] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:35.552054 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d2988-0214-4e63-865c-2e7119c28ab2 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:35.565963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d2988-0214-4e63-865c-2e7119c28ab2 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:35.572585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63bec7-80f9-4969-bca4-a5315306be8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:35.574742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63bec7-80f9-4969-bca4-a5315306be8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:35.574949 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63bec7-80f9-4969-bca4-a5315306be8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:35.575144 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63bec7-80f9-4969-bca4-a5315306be8a admin admin] PATH_INFO: `/v3/projects/d9d88964fca14a2e978710772e23a598/users/3891ff1209124d92864a9230b1229feb/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:35.593264 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63bec7-80f9-4969-bca4-a5315306be8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9d88964fca14a2e978710772e23a598, user_id=3891ff1209124d92864a9230b1229feb, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:35.596293 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63bec7-80f9-4969-bca4-a5315306be8a admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:35.632487 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:35.725533 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 976/1933] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:35 2022] PUT /identity/v3/projects/d9d88964fca14a2e978710772e23a598/users/3891ff1209124d92864a9230b1229feb/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:35.736349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a8987-519b-4f67-90a6-14d7d12d753a None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:35.736645 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a8987-519b-4f67-90a6-14d7d12d753a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:35.737091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a8987-519b-4f67-90a6-14d7d12d753a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:35.766878 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-ef6a8987-519b-4f67-90a6-14d7d12d753a None None] MFA Rules not processed for user `3891ff1209124d92864a9230b1229feb`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:35.768297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ef6a8987-519b-4f67-90a6-14d7d12d753a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:35.856359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 958/1934] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:35.861148 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63bec7-80f9-4969-bca4-a5315306be8a admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:35.947124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63bec7-80f9-4969-bca4-a5315306be8a admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:35.952792 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a9491-8d52-4f68-9487-116bf4593170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:35.953586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a9491-8d52-4f68-9487-116bf4593170 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:35.953881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a9491-8d52-4f68-9487-116bf4593170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:35.954191 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a9491-8d52-4f68-9487-116bf4593170 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:35.955026 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a9491-8d52-4f68-9487-116bf4593170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:35.957255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a9491-8d52-4f68-9487-116bf4593170 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:36.076298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a8987-519b-4f67-90a6-14d7d12d753a None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:36.084741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 977/1935] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:35 2022] POST /identity/v3/projects => generated 377 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:36.090402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a8987-519b-4f67-90a6-14d7d12d753a None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:36.168834 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10cdcf-3ad7-4599-87c4-f174805e1404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:36.175109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10cdcf-3ad7-4599-87c4-f174805e1404 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:36.175583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10cdcf-3ad7-4599-87c4-f174805e1404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:36.176276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10cdcf-3ad7-4599-87c4-f174805e1404 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:36.178612 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10cdcf-3ad7-4599-87c4-f174805e1404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:36.181062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10cdcf-3ad7-4599-87c4-f174805e1404 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:36.205386 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a9491-8d52-4f68-9487-116bf4593170 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:36.207469 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-749a9491-8d52-4f68-9487-116bf4593170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:36.227237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a9491-8d52-4f68-9487-116bf4593170 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:36.233485 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938eea17-7b44-4fff-a0f6-a9e9e90aa13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:36.234609 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938eea17-7b44-4fff-a0f6-a9e9e90aa13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:36.234709 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938eea17-7b44-4fff-a0f6-a9e9e90aa13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:36.235436 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938eea17-7b44-4fff-a0f6-a9e9e90aa13b admin admin] PATH_INFO: `/v3/users/41fa4478d6484c71a78d20609617f998` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:36.236635 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938eea17-7b44-4fff-a0f6-a9e9e90aa13b admin admin] RBAC: Authorizing `identity:delete_user(user_id=41fa4478d6484c71a78d20609617f998)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:36.238760 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938eea17-7b44-4fff-a0f6-a9e9e90aa13b admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:36.331090 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 959/1936] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:36 2022] POST /identity/v3/users => generated 295 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:14:36.336559 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10cdcf-3ad7-4599-87c4-f174805e1404 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:36.347526 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10cdcf-3ad7-4599-87c4-f174805e1404 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:36.353543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d6e2d-1a14-474c-86c3-788cb69eda4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:36.354323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d6e2d-1a14-474c-86c3-788cb69eda4a admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:36.354851 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d6e2d-1a14-474c-86c3-788cb69eda4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:36.355589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d6e2d-1a14-474c-86c3-788cb69eda4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:36.356787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d6e2d-1a14-474c-86c3-788cb69eda4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:36.358717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d6e2d-1a14-474c-86c3-788cb69eda4a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:36.372130 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 960/1937] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:36.377190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d6e2d-1a14-474c-86c3-788cb69eda4a admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:36.389443 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d6e2d-1a14-474c-86c3-788cb69eda4a admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:36.394470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997f617-0727-4d34-af27-7088c5597a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:36.395534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997f617-0727-4d34-af27-7088c5597a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:36.396646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997f617-0727-4d34-af27-7088c5597a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:36.396787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997f617-0727-4d34-af27-7088c5597a7b admin admin] PATH_INFO: `/v3/projects/a075e285054647a79931ba1354224f05/users/7649e853b6424f47808de5ccc6aa4624/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:36.412288 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997f617-0727-4d34-af27-7088c5597a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a075e285054647a79931ba1354224f05, user_id=7649e853b6424f47808de5ccc6aa4624, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:36.415082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997f617-0727-4d34-af27-7088c5597a7b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:36.472453 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-938eea17-7b44-4fff-a0f6-a9e9e90aa13b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41fa4478d6484c71a78d20609617f998'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:36.480042 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-938eea17-7b44-4fff-a0f6-a9e9e90aa13b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41fa4478d6484c71a78d20609617f998'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:36.480752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-938eea17-7b44-4fff-a0f6-a9e9e90aa13b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41fa4478d6484c71a78d20609617f998'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:36.493154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-938eea17-7b44-4fff-a0f6-a9e9e90aa13b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41fa4478d6484c71a78d20609617f998'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:36.547750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:36.555884 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 961/1938] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:36 2022] PUT /identity/v3/projects/a075e285054647a79931ba1354224f05/users/7649e853b6424f47808de5ccc6aa4624/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:36.558798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997f617-0727-4d34-af27-7088c5597a7b admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:36.580837 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-938eea17-7b44-4fff-a0f6-a9e9e90aa13b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41fa4478d6484c71a78d20609617f998'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:36.589419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 978/1939] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:36 2022] DELETE /identity/v3/users/41fa4478d6484c71a78d20609617f998 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:36.596781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aac7d3-d42b-427c-9e8b-e27c73b274a6 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:36.597355 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aac7d3-d42b-427c-9e8b-e27c73b274a6 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:36.597473 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aac7d3-d42b-427c-9e8b-e27c73b274a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:36.622646 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-a2aac7d3-d42b-427c-9e8b-e27c73b274a6 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:36.623983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a2aac7d3-d42b-427c-9e8b-e27c73b274a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:36.662904 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997f617-0727-4d34-af27-7088c5597a7b admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:36.697571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 979/1940] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:36.707700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aac7d3-d42b-427c-9e8b-e27c73b274a6 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:36.710316 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a2aac7d3-d42b-427c-9e8b-e27c73b274a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:36.731153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afb967-0b8a-48dc-a76e-daef49a93e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:36.731954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afb967-0b8a-48dc-a76e-daef49a93e8e admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:36.732330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afb967-0b8a-48dc-a76e-daef49a93e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:36.732694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afb967-0b8a-48dc-a76e-daef49a93e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:36.733550 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afb967-0b8a-48dc-a76e-daef49a93e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:36.735216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afb967-0b8a-48dc-a76e-daef49a93e8e admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:36.749512 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 962/1941] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:14:36.750595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afb967-0b8a-48dc-a76e-daef49a93e8e admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:36.752329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e9afb967-0b8a-48dc-a76e-daef49a93e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:36.765374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afb967-0b8a-48dc-a76e-daef49a93e8e admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:36.774157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d638605-38fa-48d6-b8b7-da142d9d42f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:36.775221 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d638605-38fa-48d6-b8b7-da142d9d42f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:36.775505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d638605-38fa-48d6-b8b7-da142d9d42f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:36.775793 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d638605-38fa-48d6-b8b7-da142d9d42f9 admin admin] PATH_INFO: `/v3/projects/a075e285054647a79931ba1354224f05/users/7649e853b6424f47808de5ccc6aa4624/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:36.777990 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d638605-38fa-48d6-b8b7-da142d9d42f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a075e285054647a79931ba1354224f05, user_id=7649e853b6424f47808de5ccc6aa4624, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:36.779632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aac7d3-d42b-427c-9e8b-e27c73b274a6 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:36.780795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d638605-38fa-48d6-b8b7-da142d9d42f9 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:36.783782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfff82f-e913-4362-b4b9-9911f26a12eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:36.784582 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfff82f-e913-4362-b4b9-9911f26a12eb service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:36.784963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfff82f-e913-4362-b4b9-9911f26a12eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:36.788404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfff82f-e913-4362-b4b9-9911f26a12eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:36.795457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 963/1942] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:36 2022] PUT /identity/v3/projects/a075e285054647a79931ba1354224f05/users/7649e853b6424f47808de5ccc6aa4624/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:14:36.805255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfff82f-e913-4362-b4b9-9911f26a12eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:36.805255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfff82f-e913-4362-b4b9-9911f26a12eb service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:36.809004 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7fab68-e538-4f6f-9206-9d95430a2fbc None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:36.809333 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7fab68-e538-4f6f-9206-9d95430a2fbc None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:36.809878 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7fab68-e538-4f6f-9206-9d95430a2fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:36.811957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 980/1943] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:14:36.839932 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-eb7fab68-e538-4f6f-9206-9d95430a2fbc None None] MFA Rules not processed for user `7649e853b6424f47808de5ccc6aa4624`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:36.841220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-eb7fab68-e538-4f6f-9206-9d95430a2fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:36.863005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:36.934163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 964/1944] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:36.949043 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfff82f-e913-4362-b4b9-9911f26a12eb service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:36.950807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3dfff82f-e913-4362-b4b9-9911f26a12eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:37.011422 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfff82f-e913-4362-b4b9-9911f26a12eb service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:37.067023 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561bfc1-0693-46b3-a2f0-cbb3afed84a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:37.067769 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561bfc1-0693-46b3-a2f0-cbb3afed84a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:37.068964 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561bfc1-0693-46b3-a2f0-cbb3afed84a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:37.069136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561bfc1-0693-46b3-a2f0-cbb3afed84a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:37.071407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7561bfc1-0693-46b3-a2f0-cbb3afed84a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:37.144347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561bfc1-0693-46b3-a2f0-cbb3afed84a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:37.145960 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561bfc1-0693-46b3-a2f0-cbb3afed84a9 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:37.157467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 981/1945] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:14:37.448525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7fab68-e538-4f6f-9206-9d95430a2fbc None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:37.450276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-eb7fab68-e538-4f6f-9206-9d95430a2fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:37.532279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7fab68-e538-4f6f-9206-9d95430a2fbc None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:37.537787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f1784-e50b-4b32-9ff9-7096419796d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:37.538831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f1784-e50b-4b32-9ff9-7096419796d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:37.539205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f1784-e50b-4b32-9ff9-7096419796d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:37.539584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f1784-e50b-4b32-9ff9-7096419796d3 admin admin] PATH_INFO: `/v3/projects/a2022fa1ba714205ac44310bcd814210` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:37.540914 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f1784-e50b-4b32-9ff9-7096419796d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2022fa1ba714205ac44310bcd814210)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:37.543026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f1784-e50b-4b32-9ff9-7096419796d3 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:37.571125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:37.701379 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-4c2f1784-e50b-4b32-9ff9-7096419796d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2022fa1ba714205ac44310bcd814210', 'request_id': 'req-4c2f1784-e50b-4b32-9ff9-7096419796d3'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:37.707892 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-4c2f1784-e50b-4b32-9ff9-7096419796d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2022fa1ba714205ac44310bcd814210', 'request_id': 'req-4c2f1784-e50b-4b32-9ff9-7096419796d3'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:37.714437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-4c2f1784-e50b-4b32-9ff9-7096419796d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2022fa1ba714205ac44310bcd814210', 'request_id': 'req-4c2f1784-e50b-4b32-9ff9-7096419796d3'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:37.820877 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-4c2f1784-e50b-4b32-9ff9-7096419796d3 admin admin] The token cache is being invalidate because project a2022fa1ba714205ac44310bcd814210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:37.821169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-4c2f1784-e50b-4b32-9ff9-7096419796d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:37.821924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-4c2f1784-e50b-4b32-9ff9-7096419796d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:37.823748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 965/1946] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:37 2022] DELETE /identity/v3/projects/a2022fa1ba714205ac44310bcd814210 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:37.828051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561bfc1-0693-46b3-a2f0-cbb3afed84a9 service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:37.839898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561bfc1-0693-46b3-a2f0-cbb3afed84a9 service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:37.900087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f3e6f-a31a-4b17-8061-617716bf1290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:37.901082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f3e6f-a31a-4b17-8061-617716bf1290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:37.901370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f3e6f-a31a-4b17-8061-617716bf1290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:37.901689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f3e6f-a31a-4b17-8061-617716bf1290 admin admin] PATH_INFO: `/v3/users/6a26a1f816624990b64d95122dbe1880` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:37.902926 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f3e6f-a31a-4b17-8061-617716bf1290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a26a1f816624990b64d95122dbe1880)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:37.905111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f3e6f-a31a-4b17-8061-617716bf1290 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:38.093418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-211f3e6f-a31a-4b17-8061-617716bf1290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a26a1f816624990b64d95122dbe1880'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:38.106245 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-211f3e6f-a31a-4b17-8061-617716bf1290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a26a1f816624990b64d95122dbe1880'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:38.108694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-211f3e6f-a31a-4b17-8061-617716bf1290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a26a1f816624990b64d95122dbe1880'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:38.140078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-211f3e6f-a31a-4b17-8061-617716bf1290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a26a1f816624990b64d95122dbe1880'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:38.297828 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-211f3e6f-a31a-4b17-8061-617716bf1290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a26a1f816624990b64d95122dbe1880'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:38.314397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 982/1947] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:37 2022] DELETE /identity/v3/users/6a26a1f816624990b64d95122dbe1880 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:38.546606 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:38.984323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f1784-e50b-4b32-9ff9-7096419796d3 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:38.989863 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-4c2f1784-e50b-4b32-9ff9-7096419796d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:39.005864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f1784-e50b-4b32-9ff9-7096419796d3 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:39.085417 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685d829-81c8-4919-aaeb-8fe7d5ec9481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:39.086395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685d829-81c8-4919-aaeb-8fe7d5ec9481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:39.086718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685d829-81c8-4919-aaeb-8fe7d5ec9481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:39.087010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685d829-81c8-4919-aaeb-8fe7d5ec9481 admin admin] PATH_INFO: `/v3/projects/6ea8012c74e144f8b46cfa773349a348` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:39.088673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685d829-81c8-4919-aaeb-8fe7d5ec9481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ea8012c74e144f8b46cfa773349a348)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:39.090780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685d829-81c8-4919-aaeb-8fe7d5ec9481 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:39.296631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:39.298363 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c685d829-81c8-4919-aaeb-8fe7d5ec9481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ea8012c74e144f8b46cfa773349a348', 'request_id': 'req-c685d829-81c8-4919-aaeb-8fe7d5ec9481'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:39.306465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c685d829-81c8-4919-aaeb-8fe7d5ec9481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ea8012c74e144f8b46cfa773349a348', 'request_id': 'req-c685d829-81c8-4919-aaeb-8fe7d5ec9481'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:39.312512 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c685d829-81c8-4919-aaeb-8fe7d5ec9481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ea8012c74e144f8b46cfa773349a348', 'request_id': 'req-c685d829-81c8-4919-aaeb-8fe7d5ec9481'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:39.433177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c685d829-81c8-4919-aaeb-8fe7d5ec9481 admin admin] The token cache is being invalidate because project 6ea8012c74e144f8b46cfa773349a348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:39.433477 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c685d829-81c8-4919-aaeb-8fe7d5ec9481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:39.434334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c685d829-81c8-4919-aaeb-8fe7d5ec9481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:39.436045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 966/1948] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:38 2022] DELETE /identity/v3/projects/6ea8012c74e144f8b46cfa773349a348 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:39.441429 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbbbb5-e4bb-41e9-80bb-49a6c85a515d None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:39.441959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbbbb5-e4bb-41e9-80bb-49a6c85a515d None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:39.442049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbbbb5-e4bb-41e9-80bb-49a6c85a515d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:39.464499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-04fbbbb5-e4bb-41e9-80bb-49a6c85a515d None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:39.466384 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-04fbbbb5-e4bb-41e9-80bb-49a6c85a515d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:39.530500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 983/1949] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:39.539346 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c551359-a990-4e71-b15c-d4e7f95b3f35 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:39.539642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c551359-a990-4e71-b15c-d4e7f95b3f35 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:39.539931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c551359-a990-4e71-b15c-d4e7f95b3f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:39.568961 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-5c551359-a990-4e71-b15c-d4e7f95b3f35 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:39.568961 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-5c551359-a990-4e71-b15c-d4e7f95b3f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:39.582848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 967/1950] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:39.585200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbbbb5-e4bb-41e9-80bb-49a6c85a515d None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:39.606027 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbbbb5-e4bb-41e9-80bb-49a6c85a515d None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:39.612219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b467092-b313-432f-93be-28df44928b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:39.613019 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b467092-b313-432f-93be-28df44928b87 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:39.613533 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b467092-b313-432f-93be-28df44928b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:39.613891 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b467092-b313-432f-93be-28df44928b87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:39.615919 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b467092-b313-432f-93be-28df44928b87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:14:39.616564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b467092-b313-432f-93be-28df44928b87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:39.618370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b467092-b313-432f-93be-28df44928b87 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:39.621879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:39.634778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 984/1951] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:14:39.641592 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c551359-a990-4e71-b15c-d4e7f95b3f35 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:39.665758 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c551359-a990-4e71-b15c-d4e7f95b3f35 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:39.671887 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c6b2d-9d6d-4bc5-8e82-6b2a8be11142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:39.672860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c6b2d-9d6d-4bc5-8e82-6b2a8be11142 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:39.673281 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c6b2d-9d6d-4bc5-8e82-6b2a8be11142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:39.674384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c6b2d-9d6d-4bc5-8e82-6b2a8be11142 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:39.675497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c6b2d-9d6d-4bc5-8e82-6b2a8be11142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:39.680080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c6b2d-9d6d-4bc5-8e82-6b2a8be11142 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:39.814414 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 968/1952] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:39 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:39.815686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b467092-b313-432f-93be-28df44928b87 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:39.839071 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b467092-b313-432f-93be-28df44928b87 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:39.927087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060bebda-cd80-42f2-b5eb-e3f7d6dd7355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:39.928562 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bebda-cd80-42f2-b5eb-e3f7d6dd7355 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:39.928856 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bebda-cd80-42f2-b5eb-e3f7d6dd7355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:39.929989 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bebda-cd80-42f2-b5eb-e3f7d6dd7355 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:39.930876 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060bebda-cd80-42f2-b5eb-e3f7d6dd7355 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:39.933221 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060bebda-cd80-42f2-b5eb-e3f7d6dd7355 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:40.060027 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 985/1953] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:39 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:40.062983 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c6b2d-9d6d-4bc5-8e82-6b2a8be11142 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:40.073903 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c6b2d-9d6d-4bc5-8e82-6b2a8be11142 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:40.078665 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a8d53-a9d6-4a7a-9c98-badb57048853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:40.079442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a8d53-a9d6-4a7a-9c98-badb57048853 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:40.079798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a8d53-a9d6-4a7a-9c98-badb57048853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:40.080157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a8d53-a9d6-4a7a-9c98-badb57048853 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:40.080981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a8d53-a9d6-4a7a-9c98-badb57048853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:40.082840 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a8d53-a9d6-4a7a-9c98-badb57048853 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:40.095823 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 969/1954] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:40.101084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060bebda-cd80-42f2-b5eb-e3f7d6dd7355 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:40.113363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060bebda-cd80-42f2-b5eb-e3f7d6dd7355 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:40.119782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ab569-5f48-4a9d-8669-c871d5948d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:40.128397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ab569-5f48-4a9d-8669-c871d5948d88 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:40.128397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ab569-5f48-4a9d-8669-c871d5948d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:40.128397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ab569-5f48-4a9d-8669-c871d5948d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:40.128397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ab569-5f48-4a9d-8669-c871d5948d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:40.128397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ab569-5f48-4a9d-8669-c871d5948d88 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:40.137563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 986/1955] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:40.142135 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a8d53-a9d6-4a7a-9c98-badb57048853 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:40.154223 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a8d53-a9d6-4a7a-9c98-badb57048853 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:40.159071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa187909-2459-40fc-900e-fc6e9dd2d129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:40.159969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa187909-2459-40fc-900e-fc6e9dd2d129 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:40.160264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa187909-2459-40fc-900e-fc6e9dd2d129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:40.160547 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa187909-2459-40fc-900e-fc6e9dd2d129 admin admin] PATH_INFO: `/v3/projects/afe516e686394b019cf1cfda47c80218/users/a8309ebcebb1419a957852ef4df32439/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:40.190913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa187909-2459-40fc-900e-fc6e9dd2d129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afe516e686394b019cf1cfda47c80218, user_id=a8309ebcebb1419a957852ef4df32439, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:40.193390 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ab569-5f48-4a9d-8669-c871d5948d88 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:40.195425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e95ab569-5f48-4a9d-8669-c871d5948d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:40.197619 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa187909-2459-40fc-900e-fc6e9dd2d129 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:40.225931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ab569-5f48-4a9d-8669-c871d5948d88 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:40.225931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044ef5c-c76b-47b6-bb15-50a5bc68b14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:40.226723 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044ef5c-c76b-47b6-bb15-50a5bc68b14a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:40.227018 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044ef5c-c76b-47b6-bb15-50a5bc68b14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:40.227302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044ef5c-c76b-47b6-bb15-50a5bc68b14a admin admin] PATH_INFO: `/v3/users/875e46202d454079ae9d44bdacc8d077` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:40.228946 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044ef5c-c76b-47b6-bb15-50a5bc68b14a admin admin] RBAC: Authorizing `identity:delete_user(user_id=875e46202d454079ae9d44bdacc8d077)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:40.231035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044ef5c-c76b-47b6-bb15-50a5bc68b14a admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:40.284341 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 970/1956] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:40 2022] PUT /identity/v3/projects/afe516e686394b019cf1cfda47c80218/users/a8309ebcebb1419a957852ef4df32439/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:40.296341 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d493f4-8c02-4149-a26d-fc12a8427183 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:40.296653 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d493f4-8c02-4149-a26d-fc12a8427183 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:40.296922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d493f4-8c02-4149-a26d-fc12a8427183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:40.326965 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-03d493f4-8c02-4149-a26d-fc12a8427183 None None] MFA Rules not processed for user `a8309ebcebb1419a957852ef4df32439`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:40.328289 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-03d493f4-8c02-4149-a26d-fc12a8427183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:40.403588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 971/1957] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:40.410267 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d493f4-8c02-4149-a26d-fc12a8427183 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:40.449953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7044ef5c-c76b-47b6-bb15-50a5bc68b14a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '875e46202d454079ae9d44bdacc8d077'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:40.462070 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7044ef5c-c76b-47b6-bb15-50a5bc68b14a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '875e46202d454079ae9d44bdacc8d077'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:40.464741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7044ef5c-c76b-47b6-bb15-50a5bc68b14a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '875e46202d454079ae9d44bdacc8d077'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:40.482476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7044ef5c-c76b-47b6-bb15-50a5bc68b14a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '875e46202d454079ae9d44bdacc8d077'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:40.510564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d493f4-8c02-4149-a26d-fc12a8427183 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:40.515591 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214cd620-914f-4934-8dab-67c8a2db3484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:40.516329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cd620-914f-4934-8dab-67c8a2db3484 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:40.516579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cd620-914f-4934-8dab-67c8a2db3484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:40.516867 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cd620-914f-4934-8dab-67c8a2db3484 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:40.517592 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214cd620-914f-4934-8dab-67c8a2db3484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:40.519666 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214cd620-914f-4934-8dab-67c8a2db3484 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:40.580994 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7044ef5c-c76b-47b6-bb15-50a5bc68b14a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '875e46202d454079ae9d44bdacc8d077'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:40.590077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 987/1958] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:40 2022] DELETE /identity/v3/users/875e46202d454079ae9d44bdacc8d077 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:40.621122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 972/1959] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:40 2022] POST /identity/v3/projects => generated 385 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:14:40.625096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044ef5c-c76b-47b6-bb15-50a5bc68b14a admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:40.626628 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7044ef5c-c76b-47b6-bb15-50a5bc68b14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:40.707717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044ef5c-c76b-47b6-bb15-50a5bc68b14a admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:40.795136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404fcf9-edef-4931-9e81-d62ef09f62a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:40.795846 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404fcf9-edef-4931-9e81-d62ef09f62a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:40.796121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404fcf9-edef-4931-9e81-d62ef09f62a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:40.796397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404fcf9-edef-4931-9e81-d62ef09f62a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:40.797146 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404fcf9-edef-4931-9e81-d62ef09f62a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:40.799315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404fcf9-edef-4931-9e81-d62ef09f62a6 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:40.827597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:40.937849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 988/1960] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:40 2022] POST /identity/v3/users => generated 299 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:14:40.941874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214cd620-914f-4934-8dab-67c8a2db3484 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:40.954257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214cd620-914f-4934-8dab-67c8a2db3484 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:40.959780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:40.960937 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc524cd-3477-41b4-b09a-4a0140da269a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:40.961649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc524cd-3477-41b4-b09a-4a0140da269a admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:40.961910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc524cd-3477-41b4-b09a-4a0140da269a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:40.962193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc524cd-3477-41b4-b09a-4a0140da269a admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:40.962962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc524cd-3477-41b4-b09a-4a0140da269a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:40.969091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc524cd-3477-41b4-b09a-4a0140da269a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:40.974364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404fcf9-edef-4931-9e81-d62ef09f62a6 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:40.977306 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7404fcf9-edef-4931-9e81-d62ef09f62a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:40.981315 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 973/1961] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:40.985454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc524cd-3477-41b4-b09a-4a0140da269a admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:41.000372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc524cd-3477-41b4-b09a-4a0140da269a admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:41.007951 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f5592-a5c6-4703-aa9e-8119514db6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:41.009154 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f5592-a5c6-4703-aa9e-8119514db6c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:41.009439 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f5592-a5c6-4703-aa9e-8119514db6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:41.009815 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f5592-a5c6-4703-aa9e-8119514db6c3 admin admin] PATH_INFO: `/v3/projects/ef3a25dbbba2469f953951a11261f297/users/580cab27d1f84f0fbeb32f8cf98bed6b/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:41.014585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404fcf9-edef-4931-9e81-d62ef09f62a6 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:41.030274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354681b-7c20-4788-a97a-43ac452a96d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:41.031219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354681b-7c20-4788-a97a-43ac452a96d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:41.031502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354681b-7c20-4788-a97a-43ac452a96d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:41.031780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354681b-7c20-4788-a97a-43ac452a96d4 admin admin] PATH_INFO: `/v3/projects/130badbb699444e6b865087023dddf29` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:41.032178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f5592-a5c6-4703-aa9e-8119514db6c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef3a25dbbba2469f953951a11261f297, user_id=580cab27d1f84f0fbeb32f8cf98bed6b, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:41.033260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354681b-7c20-4788-a97a-43ac452a96d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=130badbb699444e6b865087023dddf29)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:41.035005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354681b-7c20-4788-a97a-43ac452a96d4 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:41.039323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f5592-a5c6-4703-aa9e-8119514db6c3 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:41.204326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 974/1962] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:40 2022] PUT /identity/v3/projects/ef3a25dbbba2469f953951a11261f297/users/580cab27d1f84f0fbeb32f8cf98bed6b/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:14:41.208320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f5592-a5c6-4703-aa9e-8119514db6c3 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:41.243663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1354681b-7c20-4788-a97a-43ac452a96d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '130badbb699444e6b865087023dddf29', 'request_id': 'req-1354681b-7c20-4788-a97a-43ac452a96d4'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:41.250693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1354681b-7c20-4788-a97a-43ac452a96d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '130badbb699444e6b865087023dddf29', 'request_id': 'req-1354681b-7c20-4788-a97a-43ac452a96d4'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:41.274921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f5592-a5c6-4703-aa9e-8119514db6c3 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:41.335646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de04df5-0b3e-4f64-adf9-56c35b56f9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:41.336331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de04df5-0b3e-4f64-adf9-56c35b56f9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:41.336622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de04df5-0b3e-4f64-adf9-56c35b56f9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:41.337009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de04df5-0b3e-4f64-adf9-56c35b56f9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:41.337645 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de04df5-0b3e-4f64-adf9-56c35b56f9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:41.339402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de04df5-0b3e-4f64-adf9-56c35b56f9a6 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:41.350590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 975/1963] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:14:41.354564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de04df5-0b3e-4f64-adf9-56c35b56f9a6 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:41.358424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1354681b-7c20-4788-a97a-43ac452a96d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '130badbb699444e6b865087023dddf29', 'request_id': 'req-1354681b-7c20-4788-a97a-43ac452a96d4'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:41.365632 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de04df5-0b3e-4f64-adf9-56c35b56f9a6 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:41.371086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdcddb-9a94-4a59-80e4-6b16d7d0523a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:41.376036 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdcddb-9a94-4a59-80e4-6b16d7d0523a admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:41.376036 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdcddb-9a94-4a59-80e4-6b16d7d0523a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:41.376036 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdcddb-9a94-4a59-80e4-6b16d7d0523a admin admin] PATH_INFO: `/v3/projects/ef3a25dbbba2469f953951a11261f297/users/580cab27d1f84f0fbeb32f8cf98bed6b/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:41.376281 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1354681b-7c20-4788-a97a-43ac452a96d4 admin admin] The token cache is being invalidate because project 130badbb699444e6b865087023dddf29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:41.376281 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1354681b-7c20-4788-a97a-43ac452a96d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:41.376516 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdcddb-9a94-4a59-80e4-6b16d7d0523a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef3a25dbbba2469f953951a11261f297, user_id=580cab27d1f84f0fbeb32f8cf98bed6b, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:41.376595 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1354681b-7c20-4788-a97a-43ac452a96d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:41.376595 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 989/1964] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:40 2022] DELETE /identity/v3/projects/130badbb699444e6b865087023dddf29 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:14:41.377243 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdcddb-9a94-4a59-80e4-6b16d7d0523a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:41.380550 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354681b-7c20-4788-a97a-43ac452a96d4 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:41.382083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1354681b-7c20-4788-a97a-43ac452a96d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:41.390282 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 976/1965] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:41 2022] PUT /identity/v3/projects/ef3a25dbbba2469f953951a11261f297/users/580cab27d1f84f0fbeb32f8cf98bed6b/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:41.401840 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354681b-7c20-4788-a97a-43ac452a96d4 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:41.402268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1f237-019b-4ae3-9489-ba88b3b18039 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:41.402599 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1f237-019b-4ae3-9489-ba88b3b18039 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:41.403149 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1f237-019b-4ae3-9489-ba88b3b18039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:41.434297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d3f1f237-019b-4ae3-9489-ba88b3b18039 None None] MFA Rules not processed for user `580cab27d1f84f0fbeb32f8cf98bed6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:41.435599 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d3f1f237-019b-4ae3-9489-ba88b3b18039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:41.475590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff182a-8f35-4301-944d-04c34d9476af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:41.476540 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff182a-8f35-4301-944d-04c34d9476af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:41.476796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff182a-8f35-4301-944d-04c34d9476af admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:41.479407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff182a-8f35-4301-944d-04c34d9476af admin admin] PATH_INFO: `/v3/users/a90ce57806964779816f3a0b542f12ef` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:41.479407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff182a-8f35-4301-944d-04c34d9476af admin admin] RBAC: Authorizing `identity:delete_user(user_id=a90ce57806964779816f3a0b542f12ef)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:41.480393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff182a-8f35-4301-944d-04c34d9476af admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:41.497618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:41.523786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 977/1966] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:41.533447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:41.534987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1f237-019b-4ae3-9489-ba88b3b18039 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:41.536519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d3f1f237-019b-4ae3-9489-ba88b3b18039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:41.602799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1f237-019b-4ae3-9489-ba88b3b18039 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:41.607280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487b096-8707-47ec-aab9-b57bf9ccdfd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:41.608176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487b096-8707-47ec-aab9-b57bf9ccdfd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:41.608608 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487b096-8707-47ec-aab9-b57bf9ccdfd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:41.609040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487b096-8707-47ec-aab9-b57bf9ccdfd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:41.611726 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-3487b096-8707-47ec-aab9-b57bf9ccdfd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:41.695653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6dff182a-8f35-4301-944d-04c34d9476af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a90ce57806964779816f3a0b542f12ef'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:41.695653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6dff182a-8f35-4301-944d-04c34d9476af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a90ce57806964779816f3a0b542f12ef'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:41.695653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6dff182a-8f35-4301-944d-04c34d9476af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a90ce57806964779816f3a0b542f12ef'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:41.708457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487b096-8707-47ec-aab9-b57bf9ccdfd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:41.712367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487b096-8707-47ec-aab9-b57bf9ccdfd3 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:41.717932 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6dff182a-8f35-4301-944d-04c34d9476af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a90ce57806964779816f3a0b542f12ef'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:41.727617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 978/1967] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:14:41.809453 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6dff182a-8f35-4301-944d-04c34d9476af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a90ce57806964779816f3a0b542f12ef'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:41.817193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 990/1968] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:41 2022] DELETE /identity/v3/users/a90ce57806964779816f3a0b542f12ef => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:14:42.236186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487b096-8707-47ec-aab9-b57bf9ccdfd3 service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:42.256487 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487b096-8707-47ec-aab9-b57bf9ccdfd3 service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:42.367337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1446991-f357-4a99-a437-da948b02e462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:42.373364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1446991-f357-4a99-a437-da948b02e462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:42.373364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1446991-f357-4a99-a437-da948b02e462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:42.373364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1446991-f357-4a99-a437-da948b02e462 admin admin] PATH_INFO: `/v3/projects/55d46b9d2ee54c13afc48f883c648d16` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:42.373364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1446991-f357-4a99-a437-da948b02e462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55d46b9d2ee54c13afc48f883c648d16)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:42.375202 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1446991-f357-4a99-a437-da948b02e462 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:42.412440 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:42.538473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c1446991-f357-4a99-a437-da948b02e462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55d46b9d2ee54c13afc48f883c648d16', 'request_id': 'req-c1446991-f357-4a99-a437-da948b02e462'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:42.550526 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c1446991-f357-4a99-a437-da948b02e462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55d46b9d2ee54c13afc48f883c648d16', 'request_id': 'req-c1446991-f357-4a99-a437-da948b02e462'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:42.559820 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c1446991-f357-4a99-a437-da948b02e462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55d46b9d2ee54c13afc48f883c648d16', 'request_id': 'req-c1446991-f357-4a99-a437-da948b02e462'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:42.651844 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c1446991-f357-4a99-a437-da948b02e462 admin admin] The token cache is being invalidate because project 55d46b9d2ee54c13afc48f883c648d16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:42.652155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c1446991-f357-4a99-a437-da948b02e462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:42.652857 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c1446991-f357-4a99-a437-da948b02e462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:42.658022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 979/1969] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:42 2022] DELETE /identity/v3/projects/55d46b9d2ee54c13afc48f883c648d16 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:42.666264 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9baba-b6af-479c-9015-ce115a26fe68 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:42.667380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9baba-b6af-479c-9015-ce115a26fe68 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:42.668053 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9baba-b6af-479c-9015-ce115a26fe68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:42.696319 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-1af9baba-b6af-479c-9015-ce115a26fe68 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:42.698385 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1af9baba-b6af-479c-9015-ce115a26fe68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:42.800207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 991/1970] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:42.810735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd44307-1d59-4589-9936-3b4daeb108c7 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:42.811087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd44307-1d59-4589-9936-3b4daeb108c7 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:42.811898 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd44307-1d59-4589-9936-3b4daeb108c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:42.840028 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-efd44307-1d59-4589-9936-3b4daeb108c7 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:42.841308 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-efd44307-1d59-4589-9936-3b4daeb108c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:42.863446 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 980/1971] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:42.867204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9baba-b6af-479c-9015-ce115a26fe68 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:42.879338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9baba-b6af-479c-9015-ce115a26fe68 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:42.885351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54780b5c-cb23-4eba-8456-f7b58691c81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:42.886107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54780b5c-cb23-4eba-8456-f7b58691c81f admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:42.886477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54780b5c-cb23-4eba-8456-f7b58691c81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:42.886871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54780b5c-cb23-4eba-8456-f7b58691c81f admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:42.887883 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54780b5c-cb23-4eba-8456-f7b58691c81f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:14:42.888211 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54780b5c-cb23-4eba-8456-f7b58691c81f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:42.891185 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54780b5c-cb23-4eba-8456-f7b58691c81f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:42.919678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 992/1972] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 15:14:42.932499 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd44307-1d59-4589-9936-3b4daeb108c7 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:42.953757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd44307-1d59-4589-9936-3b4daeb108c7 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:42.961250 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb59f9-07d0-4edd-aa97-ba3988a40704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:42.962366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb59f9-07d0-4edd-aa97-ba3988a40704 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:42.962818 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb59f9-07d0-4edd-aa97-ba3988a40704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:42.963234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb59f9-07d0-4edd-aa97-ba3988a40704 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:42.964129 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb59f9-07d0-4edd-aa97-ba3988a40704 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:42.967491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb59f9-07d0-4edd-aa97-ba3988a40704 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:43.126223 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 981/1973] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:42 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:43.131467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54780b5c-cb23-4eba-8456-f7b58691c81f admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:43.141842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54780b5c-cb23-4eba-8456-f7b58691c81f admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:43.207298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17e8ce-74b7-4b93-906d-cc9549488698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:43.207986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17e8ce-74b7-4b93-906d-cc9549488698 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:43.208268 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17e8ce-74b7-4b93-906d-cc9549488698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:43.209042 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17e8ce-74b7-4b93-906d-cc9549488698 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:43.210170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17e8ce-74b7-4b93-906d-cc9549488698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:43.212420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17e8ce-74b7-4b93-906d-cc9549488698 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:43.296408 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 993/1974] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:43 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:43.300834 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb59f9-07d0-4edd-aa97-ba3988a40704 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:43.313372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb59f9-07d0-4edd-aa97-ba3988a40704 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:43.319024 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ed6de-00bd-4b6f-bda0-d244e559b950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:43.319783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ed6de-00bd-4b6f-bda0-d244e559b950 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:43.320086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ed6de-00bd-4b6f-bda0-d244e559b950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:43.320470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ed6de-00bd-4b6f-bda0-d244e559b950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:43.321331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ed6de-00bd-4b6f-bda0-d244e559b950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:43.323050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ed6de-00bd-4b6f-bda0-d244e559b950 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:43.335968 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 982/1975] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:43.340647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17e8ce-74b7-4b93-906d-cc9549488698 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:43.347779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:43.354767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17e8ce-74b7-4b93-906d-cc9549488698 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:43.359959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821e971-e533-43e9-a45e-d01fba282412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:43.360778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821e971-e533-43e9-a45e-d01fba282412 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:43.361158 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821e971-e533-43e9-a45e-d01fba282412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:43.361505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821e971-e533-43e9-a45e-d01fba282412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:43.362437 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821e971-e533-43e9-a45e-d01fba282412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:43.364160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821e971-e533-43e9-a45e-d01fba282412 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:43.378644 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 994/1976] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:43.381381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ed6de-00bd-4b6f-bda0-d244e559b950 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:43.397190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ed6de-00bd-4b6f-bda0-d244e559b950 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:43.402992 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092f992-c172-4d4f-9780-cf163c540162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:43.404388 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092f992-c172-4d4f-9780-cf163c540162 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:43.404872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092f992-c172-4d4f-9780-cf163c540162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:43.405295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092f992-c172-4d4f-9780-cf163c540162 admin admin] PATH_INFO: `/v3/projects/e7bccf6dd33749f48f8b83a6c0d290e9/users/2cf4289bb05c409e8ea115b9f57dadd1/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:43.425195 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092f992-c172-4d4f-9780-cf163c540162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7bccf6dd33749f48f8b83a6c0d290e9, user_id=2cf4289bb05c409e8ea115b9f57dadd1, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:43.428136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092f992-c172-4d4f-9780-cf163c540162 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:43.535971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 983/1977] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:43 2022] PUT /identity/v3/projects/e7bccf6dd33749f48f8b83a6c0d290e9/users/2cf4289bb05c409e8ea115b9f57dadd1/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:43.548062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad178a-4499-4297-a61e-5d7b1b727d9a None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:43.548276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad178a-4499-4297-a61e-5d7b1b727d9a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:43.548586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad178a-4499-4297-a61e-5d7b1b727d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:43.579207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-81ad178a-4499-4297-a61e-5d7b1b727d9a None None] MFA Rules not processed for user `2cf4289bb05c409e8ea115b9f57dadd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:43.580471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-81ad178a-4499-4297-a61e-5d7b1b727d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:43.663835 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 995/1978] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:43.675555 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092f992-c172-4d4f-9780-cf163c540162 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:43.677731 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b092f992-c172-4d4f-9780-cf163c540162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:43.746620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092f992-c172-4d4f-9780-cf163c540162 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:43.750780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d941f8f-cc3b-4265-840a-fef589fb60c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:43.751437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d941f8f-cc3b-4265-840a-fef589fb60c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:43.751722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d941f8f-cc3b-4265-840a-fef589fb60c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:43.752001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d941f8f-cc3b-4265-840a-fef589fb60c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:43.761683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d941f8f-cc3b-4265-840a-fef589fb60c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:43.763333 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d941f8f-cc3b-4265-840a-fef589fb60c2 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:43.774009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 984/1979] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:43.797198 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad178a-4499-4297-a61e-5d7b1b727d9a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:43.873915 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad178a-4499-4297-a61e-5d7b1b727d9a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:43.879023 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a818a5c-8cb9-4722-a57f-5366f4e336e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:43.879919 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a818a5c-8cb9-4722-a57f-5366f4e336e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:43.880200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a818a5c-8cb9-4722-a57f-5366f4e336e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:43.881006 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a818a5c-8cb9-4722-a57f-5366f4e336e8 admin admin] PATH_INFO: `/v3/users/2cf4289bb05c409e8ea115b9f57dadd1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:43.882984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a818a5c-8cb9-4722-a57f-5366f4e336e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cf4289bb05c409e8ea115b9f57dadd1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:43.884965 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a818a5c-8cb9-4722-a57f-5366f4e336e8 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:44.150105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0a818a5c-8cb9-4722-a57f-5366f4e336e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cf4289bb05c409e8ea115b9f57dadd1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:44.158027 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0a818a5c-8cb9-4722-a57f-5366f4e336e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cf4289bb05c409e8ea115b9f57dadd1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:44.158937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0a818a5c-8cb9-4722-a57f-5366f4e336e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cf4289bb05c409e8ea115b9f57dadd1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:44.175144 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0a818a5c-8cb9-4722-a57f-5366f4e336e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cf4289bb05c409e8ea115b9f57dadd1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:44.255147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0a818a5c-8cb9-4722-a57f-5366f4e336e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cf4289bb05c409e8ea115b9f57dadd1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:44.258758 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:44.265496 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 996/1980] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:43 2022] DELETE /identity/v3/users/2cf4289bb05c409e8ea115b9f57dadd1 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:44.271896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fa006-75d9-4ed3-8ce3-930e5581ce92 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:44.272405 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fa006-75d9-4ed3-8ce3-930e5581ce92 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:44.272785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fa006-75d9-4ed3-8ce3-930e5581ce92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:44.294993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-800fa006-75d9-4ed3-8ce3-930e5581ce92 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:44.296258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-800fa006-75d9-4ed3-8ce3-930e5581ce92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:44.356607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:44.372548 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 985/1981] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:14:44.385800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a818a5c-8cb9-4722-a57f-5366f4e336e8 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:44.387776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-0a818a5c-8cb9-4722-a57f-5366f4e336e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:44.464189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fa006-75d9-4ed3-8ce3-930e5581ce92 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:44.465739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-800fa006-75d9-4ed3-8ce3-930e5581ce92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:44.496291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a818a5c-8cb9-4722-a57f-5366f4e336e8 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:44.507170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f04555-9189-4fff-bb58-7623b98cb78c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:44.508890 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f04555-9189-4fff-bb58-7623b98cb78c service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:44.509205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f04555-9189-4fff-bb58-7623b98cb78c service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:44.509523 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f04555-9189-4fff-bb58-7623b98cb78c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:44.540219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f04555-9189-4fff-bb58-7623b98cb78c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:44.543952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f04555-9189-4fff-bb58-7623b98cb78c service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:44.548441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fa006-75d9-4ed3-8ce3-930e5581ce92 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:44.553200 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba2914-4eec-488b-a268-21df4448ffe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:44.556465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba2914-4eec-488b-a268-21df4448ffe4 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:44.556768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba2914-4eec-488b-a268-21df4448ffe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:44.557053 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba2914-4eec-488b-a268-21df4448ffe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:44.563078 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-44ba2914-4eec-488b-a268-21df4448ffe4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:44.570935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 997/1982] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:44.693985 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba2914-4eec-488b-a268-21df4448ffe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:44.695497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba2914-4eec-488b-a268-21df4448ffe4 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:44.706119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 986/1983] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:45.208260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f04555-9189-4fff-bb58-7623b98cb78c service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:45.210503 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-15f04555-9189-4fff-bb58-7623b98cb78c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:45.225313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f04555-9189-4fff-bb58-7623b98cb78c service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:45.230452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b4ec2-8ba6-4808-8dba-3b0635a8d60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:45.231380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b4ec2-8ba6-4808-8dba-3b0635a8d60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:45.232185 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b4ec2-8ba6-4808-8dba-3b0635a8d60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:45.232515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b4ec2-8ba6-4808-8dba-3b0635a8d60c admin admin] PATH_INFO: `/v3/projects/e7bccf6dd33749f48f8b83a6c0d290e9` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:45.234495 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b4ec2-8ba6-4808-8dba-3b0635a8d60c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7bccf6dd33749f48f8b83a6c0d290e9)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:45.236609 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b4ec2-8ba6-4808-8dba-3b0635a8d60c admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:45.409055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c83b4ec2-8ba6-4808-8dba-3b0635a8d60c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7bccf6dd33749f48f8b83a6c0d290e9', 'request_id': 'req-c83b4ec2-8ba6-4808-8dba-3b0635a8d60c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:45.415999 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c83b4ec2-8ba6-4808-8dba-3b0635a8d60c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7bccf6dd33749f48f8b83a6c0d290e9', 'request_id': 'req-c83b4ec2-8ba6-4808-8dba-3b0635a8d60c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:45.507396 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c83b4ec2-8ba6-4808-8dba-3b0635a8d60c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7bccf6dd33749f48f8b83a6c0d290e9', 'request_id': 'req-c83b4ec2-8ba6-4808-8dba-3b0635a8d60c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:45.520430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c83b4ec2-8ba6-4808-8dba-3b0635a8d60c admin admin] The token cache is being invalidate because project e7bccf6dd33749f48f8b83a6c0d290e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:45.527313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c83b4ec2-8ba6-4808-8dba-3b0635a8d60c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:45.527313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c83b4ec2-8ba6-4808-8dba-3b0635a8d60c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:45.527313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 998/1984] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:45 2022] DELETE /identity/v3/projects/e7bccf6dd33749f48f8b83a6c0d290e9 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:14:45.528917 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ad1b4-1bd5-456f-81ff-77338280f26c None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:45.529401 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ad1b4-1bd5-456f-81ff-77338280f26c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:45.529763 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ad1b4-1bd5-456f-81ff-77338280f26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:45.549748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b86ad1b4-1bd5-456f-81ff-77338280f26c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:45.551125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b86ad1b4-1bd5-456f-81ff-77338280f26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:45.647919 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 987/1985] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:45.649884 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ef25e-b7fd-4e73-8355-ddc0c203de43 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:45.650691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ef25e-b7fd-4e73-8355-ddc0c203de43 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:45.651066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ef25e-b7fd-4e73-8355-ddc0c203de43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:45.671477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-766ef25e-b7fd-4e73-8355-ddc0c203de43 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:45.672955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-766ef25e-b7fd-4e73-8355-ddc0c203de43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:45.684594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 999/1986] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:45.696328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ad1b4-1bd5-456f-81ff-77338280f26c None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:45.714527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ad1b4-1bd5-456f-81ff-77338280f26c None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:45.724672 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d924b-60ee-49e8-9f54-7af319e0e5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:45.726161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d924b-60ee-49e8-9f54-7af319e0e5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:45.727083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d924b-60ee-49e8-9f54-7af319e0e5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:45.728764 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d924b-60ee-49e8-9f54-7af319e0e5e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:45.730693 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d924b-60ee-49e8-9f54-7af319e0e5e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:14:45.731242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d924b-60ee-49e8-9f54-7af319e0e5e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:45.733652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d924b-60ee-49e8-9f54-7af319e0e5e4 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:45.755438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 988/1987] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:14:45.768630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ef25e-b7fd-4e73-8355-ddc0c203de43 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:45.780730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ef25e-b7fd-4e73-8355-ddc0c203de43 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:45.786968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379927b9-d2e9-4ee7-8a3e-9068b629e815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:45.787654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379927b9-d2e9-4ee7-8a3e-9068b629e815 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:45.788152 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379927b9-d2e9-4ee7-8a3e-9068b629e815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:45.792516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379927b9-d2e9-4ee7-8a3e-9068b629e815 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:45.792516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379927b9-d2e9-4ee7-8a3e-9068b629e815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:45.792516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379927b9-d2e9-4ee7-8a3e-9068b629e815 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:45.956001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1000/1988] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:45 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:45.962412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d924b-60ee-49e8-9f54-7af319e0e5e4 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:45.973153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d924b-60ee-49e8-9f54-7af319e0e5e4 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:46.005999 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:46.042027 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eca45e-487f-45a4-bbe6-ec5bb5fcfd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:46.042964 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eca45e-487f-45a4-bbe6-ec5bb5fcfd99 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:46.043342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eca45e-487f-45a4-bbe6-ec5bb5fcfd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:46.043631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eca45e-487f-45a4-bbe6-ec5bb5fcfd99 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:46.044373 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eca45e-487f-45a4-bbe6-ec5bb5fcfd99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:46.046611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eca45e-487f-45a4-bbe6-ec5bb5fcfd99 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:46.205232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 989/1989] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:45 2022] POST /identity/v3/users => generated 291 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:46.210095 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379927b9-d2e9-4ee7-8a3e-9068b629e815 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:46.220546 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379927b9-d2e9-4ee7-8a3e-9068b629e815 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:46.225380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96daf0-9d04-491c-8673-ca6263847ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:46.226058 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96daf0-9d04-491c-8673-ca6263847ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:46.226346 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96daf0-9d04-491c-8673-ca6263847ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:46.226695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96daf0-9d04-491c-8673-ca6263847ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:46.227431 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96daf0-9d04-491c-8673-ca6263847ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:46.229066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96daf0-9d04-491c-8673-ca6263847ce2 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:46.240452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1001/1990] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:46.244865 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eca45e-487f-45a4-bbe6-ec5bb5fcfd99 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:46.255321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eca45e-487f-45a4-bbe6-ec5bb5fcfd99 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:46.261738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f268e-1f8b-4b2f-9e2b-a85d4aaaf920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:46.266924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f268e-1f8b-4b2f-9e2b-a85d4aaaf920 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:46.266924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f268e-1f8b-4b2f-9e2b-a85d4aaaf920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:46.266924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f268e-1f8b-4b2f-9e2b-a85d4aaaf920 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:46.266924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f268e-1f8b-4b2f-9e2b-a85d4aaaf920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:46.266924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f268e-1f8b-4b2f-9e2b-a85d4aaaf920 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:46.279084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 990/1991] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:14:46.283182 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96daf0-9d04-491c-8673-ca6263847ce2 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:46.294285 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96daf0-9d04-491c-8673-ca6263847ce2 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:46.299251 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7b021-0219-4d65-8193-43a1dc573b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:46.300181 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7b021-0219-4d65-8193-43a1dc573b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:46.300464 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7b021-0219-4d65-8193-43a1dc573b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:46.300735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7b021-0219-4d65-8193-43a1dc573b00 admin admin] PATH_INFO: `/v3/projects/1a095f39225c40bf8235823c1e8f3aa4/users/79e352b4e63e4ecda46be63f3bf33f34/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:46.321043 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7b021-0219-4d65-8193-43a1dc573b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a095f39225c40bf8235823c1e8f3aa4, user_id=79e352b4e63e4ecda46be63f3bf33f34, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:46.323927 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7b021-0219-4d65-8193-43a1dc573b00 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:46.609361 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1002/1992] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:46 2022] PUT /identity/v3/projects/1a095f39225c40bf8235823c1e8f3aa4/users/79e352b4e63e4ecda46be63f3bf33f34/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:46.622278 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9612a-790f-4540-bcc3-1635d442146f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:46.622954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9612a-790f-4540-bcc3-1635d442146f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:46.623063 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9612a-790f-4540-bcc3-1635d442146f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:46.653932 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-6ea9612a-790f-4540-bcc3-1635d442146f None None] MFA Rules not processed for user `79e352b4e63e4ecda46be63f3bf33f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:46.655219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6ea9612a-790f-4540-bcc3-1635d442146f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:46.751854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 991/1993] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:46.764561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7b021-0219-4d65-8193-43a1dc573b00 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:46.766397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-37b7b021-0219-4d65-8193-43a1dc573b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:46.797053 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9612a-790f-4540-bcc3-1635d442146f None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:46.798713 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:46.800915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6ea9612a-790f-4540-bcc3-1635d442146f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:46.860575 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7b021-0219-4d65-8193-43a1dc573b00 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:46.871985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43259c72-5905-46e5-942c-d75a0794ef46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:46.871985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43259c72-5905-46e5-942c-d75a0794ef46 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:46.871985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43259c72-5905-46e5-942c-d75a0794ef46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:46.871985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43259c72-5905-46e5-942c-d75a0794ef46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:46.880779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43259c72-5905-46e5-942c-d75a0794ef46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:46.882915 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43259c72-5905-46e5-942c-d75a0794ef46 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:46.900988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1003/1994] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:46.916519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9612a-790f-4540-bcc3-1635d442146f None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:46.921480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa53de2-25ea-41d5-a980-b3a5d886be9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:46.922640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa53de2-25ea-41d5-a980-b3a5d886be9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:46.928276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa53de2-25ea-41d5-a980-b3a5d886be9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:46.928276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa53de2-25ea-41d5-a980-b3a5d886be9e admin admin] PATH_INFO: `/v3/users/41e480fc338449d2a51e13cc36ebe729` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:46.928276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa53de2-25ea-41d5-a980-b3a5d886be9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=41e480fc338449d2a51e13cc36ebe729)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:46.928276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa53de2-25ea-41d5-a980-b3a5d886be9e admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:47.256893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:47.269911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-afa53de2-25ea-41d5-a980-b3a5d886be9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41e480fc338449d2a51e13cc36ebe729'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:47.271735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-afa53de2-25ea-41d5-a980-b3a5d886be9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41e480fc338449d2a51e13cc36ebe729'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:47.272894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-afa53de2-25ea-41d5-a980-b3a5d886be9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41e480fc338449d2a51e13cc36ebe729'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:47.347652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-afa53de2-25ea-41d5-a980-b3a5d886be9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41e480fc338449d2a51e13cc36ebe729'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:47.366415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-afa53de2-25ea-41d5-a980-b3a5d886be9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41e480fc338449d2a51e13cc36ebe729'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:47.382465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 992/1995] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:46 2022] DELETE /identity/v3/users/41e480fc338449d2a51e13cc36ebe729 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:47.387685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31983b68-cd84-4439-87f3-375ead40178f None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:47.388196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31983b68-cd84-4439-87f3-375ead40178f None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:47.388321 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31983b68-cd84-4439-87f3-375ead40178f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:47.407518 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-31983b68-cd84-4439-87f3-375ead40178f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:47.408757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-31983b68-cd84-4439-87f3-375ead40178f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:47.478577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1004/1996] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:47.491377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa53de2-25ea-41d5-a980-b3a5d886be9e admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:47.493493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-afa53de2-25ea-41d5-a980-b3a5d886be9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:47.565381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa53de2-25ea-41d5-a980-b3a5d886be9e admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:47.569737 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2d58f-79c9-4735-aa01-c0a4caca346e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:47.571524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2d58f-79c9-4735-aa01-c0a4caca346e service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:47.571524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2d58f-79c9-4735-aa01-c0a4caca346e service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:47.571524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2d58f-79c9-4735-aa01-c0a4caca346e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:47.580724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2d58f-79c9-4735-aa01-c0a4caca346e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:47.582180 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2d58f-79c9-4735-aa01-c0a4caca346e service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:47.594043 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 993/1997] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:48.200299 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31983b68-cd84-4439-87f3-375ead40178f None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:48.202471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-31983b68-cd84-4439-87f3-375ead40178f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:48.222481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31983b68-cd84-4439-87f3-375ead40178f None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:48.228774 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812921eb-82d0-408f-a022-ed640b077588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:48.229673 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812921eb-82d0-408f-a022-ed640b077588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:48.229963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812921eb-82d0-408f-a022-ed640b077588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:48.230245 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812921eb-82d0-408f-a022-ed640b077588 admin admin] PATH_INFO: `/v3/projects/402857c697be47abb330793e851f6f45` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:48.231734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812921eb-82d0-408f-a022-ed640b077588 admin admin] RBAC: Authorizing `identity:delete_project(project_id=402857c697be47abb330793e851f6f45)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:48.249180 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812921eb-82d0-408f-a022-ed640b077588 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:48.397777 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-812921eb-82d0-408f-a022-ed640b077588 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '402857c697be47abb330793e851f6f45', 'request_id': 'req-812921eb-82d0-408f-a022-ed640b077588'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:48.404468 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-812921eb-82d0-408f-a022-ed640b077588 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '402857c697be47abb330793e851f6f45', 'request_id': 'req-812921eb-82d0-408f-a022-ed640b077588'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:48.495665 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-812921eb-82d0-408f-a022-ed640b077588 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '402857c697be47abb330793e851f6f45', 'request_id': 'req-812921eb-82d0-408f-a022-ed640b077588'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:48.508678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-812921eb-82d0-408f-a022-ed640b077588 admin admin] The token cache is being invalidate because project 402857c697be47abb330793e851f6f45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:48.508965 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-812921eb-82d0-408f-a022-ed640b077588 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:48.509629 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-812921eb-82d0-408f-a022-ed640b077588 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:48.511259 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1005/1998] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:48 2022] DELETE /identity/v3/projects/402857c697be47abb330793e851f6f45 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:48.520233 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26393b50-83f5-4e5e-92f8-1d84e873b9fc None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:48.520233 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26393b50-83f5-4e5e-92f8-1d84e873b9fc None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:48.520233 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26393b50-83f5-4e5e-92f8-1d84e873b9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:48.540535 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-26393b50-83f5-4e5e-92f8-1d84e873b9fc None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:48.541883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-26393b50-83f5-4e5e-92f8-1d84e873b9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:48.605904 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 994/1999] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:48.613744 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33fefc-44e4-49b3-a5de-adaaa2af8138 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:48.614041 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33fefc-44e4-49b3-a5de-adaaa2af8138 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:48.614332 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33fefc-44e4-49b3-a5de-adaaa2af8138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:48.633792 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-3f33fefc-44e4-49b3-a5de-adaaa2af8138 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:48.635061 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3f33fefc-44e4-49b3-a5de-adaaa2af8138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:48.643617 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1006/2000] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:48.648045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26393b50-83f5-4e5e-92f8-1d84e873b9fc None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:48.657943 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26393b50-83f5-4e5e-92f8-1d84e873b9fc None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:48.662607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a805f46-022e-46a7-b86c-9627ab5e2c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:48.663269 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a805f46-022e-46a7-b86c-9627ab5e2c48 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:48.663552 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a805f46-022e-46a7-b86c-9627ab5e2c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:48.663828 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a805f46-022e-46a7-b86c-9627ab5e2c48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:48.664579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a805f46-022e-46a7-b86c-9627ab5e2c48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:14:48.664886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a805f46-022e-46a7-b86c-9627ab5e2c48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:48.666555 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a805f46-022e-46a7-b86c-9627ab5e2c48 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:48.679048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 995/2001] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:14:48.683980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33fefc-44e4-49b3-a5de-adaaa2af8138 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:48.694761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33fefc-44e4-49b3-a5de-adaaa2af8138 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:48.699556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5ec2a-6ca5-4e32-87f0-6b5a97557d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:48.700231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5ec2a-6ca5-4e32-87f0-6b5a97557d18 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:48.700516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5ec2a-6ca5-4e32-87f0-6b5a97557d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:48.700791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5ec2a-6ca5-4e32-87f0-6b5a97557d18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:48.701489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5ec2a-6ca5-4e32-87f0-6b5a97557d18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:48.703542 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5ec2a-6ca5-4e32-87f0-6b5a97557d18 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:48.812771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1007/2002] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:48 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:14:48.817357 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a805f46-022e-46a7-b86c-9627ab5e2c48 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:48.828739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a805f46-022e-46a7-b86c-9627ab5e2c48 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:48.875768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:48.891766 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ab140-3e02-4010-a1ce-0b0c1af3a53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:48.892454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ab140-3e02-4010-a1ce-0b0c1af3a53d admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:48.892746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ab140-3e02-4010-a1ce-0b0c1af3a53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:48.893032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ab140-3e02-4010-a1ce-0b0c1af3a53d admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:48.893743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ab140-3e02-4010-a1ce-0b0c1af3a53d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:48.895752 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ab140-3e02-4010-a1ce-0b0c1af3a53d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:49.004793 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 996/2003] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:48 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:49.010128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5ec2a-6ca5-4e32-87f0-6b5a97557d18 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:49.020388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5ec2a-6ca5-4e32-87f0-6b5a97557d18 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:49.025408 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42624455-59d2-40c1-8df0-c74530d97b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:49.026142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42624455-59d2-40c1-8df0-c74530d97b45 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:49.031511 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42624455-59d2-40c1-8df0-c74530d97b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:49.031511 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42624455-59d2-40c1-8df0-c74530d97b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:49.031511 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42624455-59d2-40c1-8df0-c74530d97b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:49.031511 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42624455-59d2-40c1-8df0-c74530d97b45 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:49.042430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1008/2004] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:14:49.047738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ab140-3e02-4010-a1ce-0b0c1af3a53d admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:49.058960 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ab140-3e02-4010-a1ce-0b0c1af3a53d admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:49.063781 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dfb31-9ef5-4a09-944c-91bb70e04e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:49.064461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dfb31-9ef5-4a09-944c-91bb70e04e34 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:49.064748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dfb31-9ef5-4a09-944c-91bb70e04e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:49.065033 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dfb31-9ef5-4a09-944c-91bb70e04e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:49.065781 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4dfb31-9ef5-4a09-944c-91bb70e04e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:49.067457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4dfb31-9ef5-4a09-944c-91bb70e04e34 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:49.079389 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 997/2005] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:49.083857 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42624455-59d2-40c1-8df0-c74530d97b45 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:49.094796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42624455-59d2-40c1-8df0-c74530d97b45 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:49.100229 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666f47a-02d6-45d0-aa57-6a5e168252df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:49.103860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666f47a-02d6-45d0-aa57-6a5e168252df admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:49.103860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666f47a-02d6-45d0-aa57-6a5e168252df admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:49.103860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666f47a-02d6-45d0-aa57-6a5e168252df admin admin] PATH_INFO: `/v3/projects/afc824f35b8144398ace513ad018ec98/users/db821aa375c2475181fd0c9d5807a51b/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:49.105177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:49.118032 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666f47a-02d6-45d0-aa57-6a5e168252df admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc824f35b8144398ace513ad018ec98, user_id=db821aa375c2475181fd0c9d5807a51b, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:49.120776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666f47a-02d6-45d0-aa57-6a5e168252df admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:49.244811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1009/2006] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:49 2022] PUT /identity/v3/projects/afc824f35b8144398ace513ad018ec98/users/db821aa375c2475181fd0c9d5807a51b/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:49.256563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12502e-12cd-4e2d-b95b-2929ae9f8ada None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:49.256817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12502e-12cd-4e2d-b95b-2929ae9f8ada None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:49.257104 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12502e-12cd-4e2d-b95b-2929ae9f8ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:49.286591 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-0d12502e-12cd-4e2d-b95b-2929ae9f8ada None None] MFA Rules not processed for user `db821aa375c2475181fd0c9d5807a51b`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:49.288987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0d12502e-12cd-4e2d-b95b-2929ae9f8ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:49.369106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 998/2007] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:49.384590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666f47a-02d6-45d0-aa57-6a5e168252df admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:49.388664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-0666f47a-02d6-45d0-aa57-6a5e168252df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:49.477427 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666f47a-02d6-45d0-aa57-6a5e168252df admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:49.481352 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42e4b9-9e4f-47fb-9007-8439079782ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:49.482045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42e4b9-9e4f-47fb-9007-8439079782ac service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:49.482448 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42e4b9-9e4f-47fb-9007-8439079782ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:49.482764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42e4b9-9e4f-47fb-9007-8439079782ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:49.495618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42e4b9-9e4f-47fb-9007-8439079782ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:49.497151 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42e4b9-9e4f-47fb-9007-8439079782ac service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:49.508966 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1010/2008] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:14:51.014106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12502e-12cd-4e2d-b95b-2929ae9f8ada None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:51.016635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0d12502e-12cd-4e2d-b95b-2929ae9f8ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:51.030112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12502e-12cd-4e2d-b95b-2929ae9f8ada None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:51.033724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a0808-319a-463b-be26-4088a8af3a44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:51.034372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a0808-319a-463b-be26-4088a8af3a44 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:51.034698 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a0808-319a-463b-be26-4088a8af3a44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:51.034972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a0808-319a-463b-be26-4088a8af3a44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:51.036843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-852a0808-319a-463b-be26-4088a8af3a44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:51.121667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a0808-319a-463b-be26-4088a8af3a44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:51.123222 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a0808-319a-463b-be26-4088a8af3a44 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:51.136194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 999/2009] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:51.583225 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25fedc-792e-4643-b0c6-01987a439055 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:51.583943 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25fedc-792e-4643-b0c6-01987a439055 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:51.584086 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25fedc-792e-4643-b0c6-01987a439055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:51.603369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-8e25fedc-792e-4643-b0c6-01987a439055 None None] MFA Rules not processed for user `db821aa375c2475181fd0c9d5807a51b`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:51.604670 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-8e25fedc-792e-4643-b0c6-01987a439055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:51.612087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1011/2010] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:52.639375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a0808-319a-463b-be26-4088a8af3a44 service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:52.640881 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-852a0808-319a-463b-be26-4088a8af3a44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:52.691133 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:52.711683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a0808-319a-463b-be26-4088a8af3a44 service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:52.717154 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd9f36-119c-48c8-bd70-1ac4f70112e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:52.718052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd9f36-119c-48c8-bd70-1ac4f70112e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:52.718340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd9f36-119c-48c8-bd70-1ac4f70112e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:52.718644 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd9f36-119c-48c8-bd70-1ac4f70112e8 admin admin] PATH_INFO: `/v3/users/79e352b4e63e4ecda46be63f3bf33f34` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:52.720160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd9f36-119c-48c8-bd70-1ac4f70112e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79e352b4e63e4ecda46be63f3bf33f34)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:52.722088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd9f36-119c-48c8-bd70-1ac4f70112e8 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:53.034514 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a9fd9f36-119c-48c8-bd70-1ac4f70112e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79e352b4e63e4ecda46be63f3bf33f34'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:53.035721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a9fd9f36-119c-48c8-bd70-1ac4f70112e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79e352b4e63e4ecda46be63f3bf33f34'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:53.036810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a9fd9f36-119c-48c8-bd70-1ac4f70112e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79e352b4e63e4ecda46be63f3bf33f34'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:53.148655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a9fd9f36-119c-48c8-bd70-1ac4f70112e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79e352b4e63e4ecda46be63f3bf33f34'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:53.166826 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a9fd9f36-119c-48c8-bd70-1ac4f70112e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79e352b4e63e4ecda46be63f3bf33f34'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:53.182333 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1000/2011] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:52 2022] DELETE /identity/v3/users/79e352b4e63e4ecda46be63f3bf33f34 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:53.184247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26233ac7-4ed3-4609-a08d-b6b841daed89 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:53.184743 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26233ac7-4ed3-4609-a08d-b6b841daed89 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:53.184972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26233ac7-4ed3-4609-a08d-b6b841daed89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:53.203301 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-26233ac7-4ed3-4609-a08d-b6b841daed89 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:53.204539 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-26233ac7-4ed3-4609-a08d-b6b841daed89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:53.278790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1012/2012] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:53.290263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd9f36-119c-48c8-bd70-1ac4f70112e8 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:53.292623 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a9fd9f36-119c-48c8-bd70-1ac4f70112e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:53.406823 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd9f36-119c-48c8-bd70-1ac4f70112e8 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:53.410993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b284c74-9312-408b-8bf0-329b361dba5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:53.411891 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b284c74-9312-408b-8bf0-329b361dba5e service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:53.413133 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b284c74-9312-408b-8bf0-329b361dba5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:53.413378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b284c74-9312-408b-8bf0-329b361dba5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:53.424879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b284c74-9312-408b-8bf0-329b361dba5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:53.426781 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b284c74-9312-408b-8bf0-329b361dba5e service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:53.455308 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1001/2013] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:53.949330 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26233ac7-4ed3-4609-a08d-b6b841daed89 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:53.950830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-26233ac7-4ed3-4609-a08d-b6b841daed89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:53.963797 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26233ac7-4ed3-4609-a08d-b6b841daed89 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:53.968513 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece59d2b-4f53-4bd0-a4c3-d1aa372529b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:53.969393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece59d2b-4f53-4bd0-a4c3-d1aa372529b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:53.969682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece59d2b-4f53-4bd0-a4c3-d1aa372529b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:53.969960 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece59d2b-4f53-4bd0-a4c3-d1aa372529b2 admin admin] PATH_INFO: `/v3/projects/1a095f39225c40bf8235823c1e8f3aa4` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:53.971200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece59d2b-4f53-4bd0-a4c3-d1aa372529b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a095f39225c40bf8235823c1e8f3aa4)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:53.973224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece59d2b-4f53-4bd0-a4c3-d1aa372529b2 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:54.098059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ece59d2b-4f53-4bd0-a4c3-d1aa372529b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a095f39225c40bf8235823c1e8f3aa4', 'request_id': 'req-ece59d2b-4f53-4bd0-a4c3-d1aa372529b2'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:54.105953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ece59d2b-4f53-4bd0-a4c3-d1aa372529b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a095f39225c40bf8235823c1e8f3aa4', 'request_id': 'req-ece59d2b-4f53-4bd0-a4c3-d1aa372529b2'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:54.198685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ece59d2b-4f53-4bd0-a4c3-d1aa372529b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a095f39225c40bf8235823c1e8f3aa4', 'request_id': 'req-ece59d2b-4f53-4bd0-a4c3-d1aa372529b2'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:54.211058 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ece59d2b-4f53-4bd0-a4c3-d1aa372529b2 admin admin] The token cache is being invalidate because project 1a095f39225c40bf8235823c1e8f3aa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:14:54.211353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ece59d2b-4f53-4bd0-a4c3-d1aa372529b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:54.212054 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ece59d2b-4f53-4bd0-a4c3-d1aa372529b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:54.213671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1013/2014] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:14:53 2022] DELETE /identity/v3/projects/1a095f39225c40bf8235823c1e8f3aa4 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:14:54.220077 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10ecb0-eacb-4cdc-a5a5-4743b90ddd19 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:54.220613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10ecb0-eacb-4cdc-a5a5-4743b90ddd19 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:54.220613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10ecb0-eacb-4cdc-a5a5-4743b90ddd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:54.239369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-bb10ecb0-eacb-4cdc-a5a5-4743b90ddd19 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:54.241276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-bb10ecb0-eacb-4cdc-a5a5-4743b90ddd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:54.301149 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1002/2015] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:54.310068 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e070a-f9eb-4986-823b-4ea63051c236 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:54.310356 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e070a-f9eb-4986-823b-4ea63051c236 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:54.310677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e070a-f9eb-4986-823b-4ea63051c236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:54.318663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:54.329281 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-660e070a-f9eb-4986-823b-4ea63051c236 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:54.330492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-660e070a-f9eb-4986-823b-4ea63051c236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:54.338893 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1014/2016] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:54.351744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10ecb0-eacb-4cdc-a5a5-4743b90ddd19 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:54.361743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10ecb0-eacb-4cdc-a5a5-4743b90ddd19 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:54.367576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575a260-8d09-40f6-82a3-b6781d1c9b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:54.367841 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575a260-8d09-40f6-82a3-b6781d1c9b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:54.368292 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575a260-8d09-40f6-82a3-b6781d1c9b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:54.368572 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575a260-8d09-40f6-82a3-b6781d1c9b78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:54.369315 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575a260-8d09-40f6-82a3-b6781d1c9b78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:14:54.369648 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575a260-8d09-40f6-82a3-b6781d1c9b78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:54.371655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575a260-8d09-40f6-82a3-b6781d1c9b78 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:54.374569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:54.386699 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1003/2017] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:14:54.391815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e070a-f9eb-4986-823b-4ea63051c236 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:54.403740 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e070a-f9eb-4986-823b-4ea63051c236 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:54.408634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f0bb2-d9c9-443f-bf53-eb7d316e28a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:54.409304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f0bb2-d9c9-443f-bf53-eb7d316e28a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:54.409594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f0bb2-d9c9-443f-bf53-eb7d316e28a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:54.410299 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f0bb2-d9c9-443f-bf53-eb7d316e28a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:54.410601 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f0bb2-d9c9-443f-bf53-eb7d316e28a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:54.413038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f0bb2-d9c9-443f-bf53-eb7d316e28a0 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:54.560911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1015/2018] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:14:54 2022] POST /identity/v3/projects => generated 383 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:54.565530 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575a260-8d09-40f6-82a3-b6781d1c9b78 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:54.576174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575a260-8d09-40f6-82a3-b6781d1c9b78 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:54.636153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10faeb94-4046-447c-b8b2-f460156a5dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:54.636882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faeb94-4046-447c-b8b2-f460156a5dbc admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:54.637388 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faeb94-4046-447c-b8b2-f460156a5dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:54.637840 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faeb94-4046-447c-b8b2-f460156a5dbc admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:54.639014 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10faeb94-4046-447c-b8b2-f460156a5dbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:54.641105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10faeb94-4046-447c-b8b2-f460156a5dbc admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:54.772884 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1004/2019] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:14:54 2022] POST /identity/v3/users => generated 292 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:14:54.777387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f0bb2-d9c9-443f-bf53-eb7d316e28a0 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:54.790173 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f0bb2-d9c9-443f-bf53-eb7d316e28a0 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:54.795925 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ef85b-1f16-4a13-a3ec-428221530f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:54.796657 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ef85b-1f16-4a13-a3ec-428221530f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:54.796947 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ef85b-1f16-4a13-a3ec-428221530f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:54.797234 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ef85b-1f16-4a13-a3ec-428221530f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:54.797981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ef85b-1f16-4a13-a3ec-428221530f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:54.799686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ef85b-1f16-4a13-a3ec-428221530f65 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:54.811553 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1016/2020] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:54.816657 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10faeb94-4046-447c-b8b2-f460156a5dbc admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:54.830402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10faeb94-4046-447c-b8b2-f460156a5dbc admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:54.836339 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d86c1d-b0a6-4300-a762-3144eb730dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:54.837144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d86c1d-b0a6-4300-a762-3144eb730dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:54.837515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d86c1d-b0a6-4300-a762-3144eb730dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:54.837861 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d86c1d-b0a6-4300-a762-3144eb730dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:54.838728 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d86c1d-b0a6-4300-a762-3144eb730dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:54.840438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d86c1d-b0a6-4300-a762-3144eb730dd7 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:54.854063 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1005/2021] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:14:54.856631 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ef85b-1f16-4a13-a3ec-428221530f65 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:54.868716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ef85b-1f16-4a13-a3ec-428221530f65 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:54.873393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5052c-172b-4e92-b36a-df47bd965934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:54.874292 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5052c-172b-4e92-b36a-df47bd965934 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:54.874616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5052c-172b-4e92-b36a-df47bd965934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:54.874899 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5052c-172b-4e92-b36a-df47bd965934 admin admin] PATH_INFO: `/v3/projects/eb8e493ee52b4c2686d5010eed312191/users/45b1df340f8a4102adf5f680d7e64139/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:54.892895 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5052c-172b-4e92-b36a-df47bd965934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb8e493ee52b4c2686d5010eed312191, user_id=45b1df340f8a4102adf5f680d7e64139, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:54.896024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5052c-172b-4e92-b36a-df47bd965934 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:54.970685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1017/2022] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:14:54 2022] PUT /identity/v3/projects/eb8e493ee52b4c2686d5010eed312191/users/45b1df340f8a4102adf5f680d7e64139/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:54.976339 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d043a1-5e21-4f66-a2a7-0531cd8c514b None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:54.976647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d043a1-5e21-4f66-a2a7-0531cd8c514b None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:54.976929 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d043a1-5e21-4f66-a2a7-0531cd8c514b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:54.996843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-20d043a1-5e21-4f66-a2a7-0531cd8c514b None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:54.998106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-20d043a1-5e21-4f66-a2a7-0531cd8c514b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:55.061336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1006/2023] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:14:55.071935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5052c-172b-4e92-b36a-df47bd965934 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:55.073700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-9de5052c-172b-4e92-b36a-df47bd965934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:55.135705 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5052c-172b-4e92-b36a-df47bd965934 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:55.139499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a188b-7c80-4435-9145-270278129746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:55.140179 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a188b-7c80-4435-9145-270278129746 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:55.140478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a188b-7c80-4435-9145-270278129746 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:55.140743 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a188b-7c80-4435-9145-270278129746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:55.150864 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a188b-7c80-4435-9145-270278129746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:55.152320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a188b-7c80-4435-9145-270278129746 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:55.162043 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1018/2024] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:14:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:14:59.339326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d043a1-5e21-4f66-a2a7-0531cd8c514b None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:14:59.341056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-20d043a1-5e21-4f66-a2a7-0531cd8c514b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:14:59.362426 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d043a1-5e21-4f66-a2a7-0531cd8c514b None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:14:59.369944 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1198ee5-d00a-4ae6-a7cc-835790091256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:14:59.371270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1198ee5-d00a-4ae6-a7cc-835790091256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:59.371708 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1198ee5-d00a-4ae6-a7cc-835790091256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:59.372104 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1198ee5-d00a-4ae6-a7cc-835790091256 admin admin] PATH_INFO: `/v3/users/a8309ebcebb1419a957852ef4df32439` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:59.373969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1198ee5-d00a-4ae6-a7cc-835790091256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8309ebcebb1419a957852ef4df32439)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:14:59.376546 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1198ee5-d00a-4ae6-a7cc-835790091256 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:14:59.700781 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-e1198ee5-d00a-4ae6-a7cc-835790091256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8309ebcebb1419a957852ef4df32439'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:59.701632 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-e1198ee5-d00a-4ae6-a7cc-835790091256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8309ebcebb1419a957852ef4df32439'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:59.702504 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-e1198ee5-d00a-4ae6-a7cc-835790091256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8309ebcebb1419a957852ef4df32439'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:59.817679 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-e1198ee5-d00a-4ae6-a7cc-835790091256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8309ebcebb1419a957852ef4df32439'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:59.860356 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:14:59.864646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-e1198ee5-d00a-4ae6-a7cc-835790091256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8309ebcebb1419a957852ef4df32439'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:14:59.906006 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1007/2025] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:14:59 2022] DELETE /identity/v3/users/a8309ebcebb1419a957852ef4df32439 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:14:59.916659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d60082-7fed-455e-a466-35b6c8212f97 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:14:59.917466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d60082-7fed-455e-a466-35b6c8212f97 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:14:59.918292 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d60082-7fed-455e-a466-35b6c8212f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:14:59.950167 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-f8d60082-7fed-455e-a466-35b6c8212f97 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:14:59.951302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f8d60082-7fed-455e-a466-35b6c8212f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:00.055514 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1019/2026] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:00.067912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1198ee5-d00a-4ae6-a7cc-835790091256 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:00.076584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e1198ee5-d00a-4ae6-a7cc-835790091256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:00.155251 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1198ee5-d00a-4ae6-a7cc-835790091256 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:00.159899 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def433a2-ba24-41fc-ac15-18039884b269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:00.160233 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def433a2-ba24-41fc-ac15-18039884b269 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:00.160683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def433a2-ba24-41fc-ac15-18039884b269 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:00.160981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def433a2-ba24-41fc-ac15-18039884b269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:00.173453 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def433a2-ba24-41fc-ac15-18039884b269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:00.175613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def433a2-ba24-41fc-ac15-18039884b269 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:00.188698 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1008/2027] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:01.141470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d60082-7fed-455e-a466-35b6c8212f97 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:01.143258 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f8d60082-7fed-455e-a466-35b6c8212f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:01.160257 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d60082-7fed-455e-a466-35b6c8212f97 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:01.172318 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:01.173733 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:01.173733 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:01.173733 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f admin admin] PATH_INFO: `/v3/projects/afe516e686394b019cf1cfda47c80218` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:01.179191 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=afe516e686394b019cf1cfda47c80218)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:01.181980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:01.352891 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:01.397003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afe516e686394b019cf1cfda47c80218', 'request_id': 'req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:01.408136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afe516e686394b019cf1cfda47c80218', 'request_id': 'req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:01.525557 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afe516e686394b019cf1cfda47c80218', 'request_id': 'req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:01.592240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f admin admin] The token cache is being invalidate because project afe516e686394b019cf1cfda47c80218 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:15:01.592757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:01.596463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:01.598608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1020/2028] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:15:01 2022] DELETE /identity/v3/projects/afe516e686394b019cf1cfda47c80218 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:01.605252 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def433a2-ba24-41fc-ac15-18039884b269 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:01.619564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def433a2-ba24-41fc-ac15-18039884b269 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:01.727391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3664d5-34cd-40bb-b210-d20a0b26b116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:01.730804 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3664d5-34cd-40bb-b210-d20a0b26b116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:01.731104 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3664d5-34cd-40bb-b210-d20a0b26b116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:01.731391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3664d5-34cd-40bb-b210-d20a0b26b116 admin admin] PATH_INFO: `/v3/users/580cab27d1f84f0fbeb32f8cf98bed6b` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:01.733464 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3664d5-34cd-40bb-b210-d20a0b26b116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=580cab27d1f84f0fbeb32f8cf98bed6b)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:01.735855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3664d5-34cd-40bb-b210-d20a0b26b116 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:01.955766 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-6f3664d5-34cd-40bb-b210-d20a0b26b116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '580cab27d1f84f0fbeb32f8cf98bed6b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:01.956469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-6f3664d5-34cd-40bb-b210-d20a0b26b116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '580cab27d1f84f0fbeb32f8cf98bed6b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:01.957378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-6f3664d5-34cd-40bb-b210-d20a0b26b116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '580cab27d1f84f0fbeb32f8cf98bed6b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:02.061661 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-6f3664d5-34cd-40bb-b210-d20a0b26b116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '580cab27d1f84f0fbeb32f8cf98bed6b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:02.089935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-6f3664d5-34cd-40bb-b210-d20a0b26b116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '580cab27d1f84f0fbeb32f8cf98bed6b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:02.127171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1009/2029] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:15:01 2022] DELETE /identity/v3/users/580cab27d1f84f0fbeb32f8cf98bed6b => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:02.853651 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:02.855256 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:02.903037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa96ba-c94d-4abc-ae8a-5aff2964eb1f admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:03.007091 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381660e-735c-4a1f-915b-582194811e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:03.012088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381660e-735c-4a1f-915b-582194811e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:03.012088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381660e-735c-4a1f-915b-582194811e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:03.012088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381660e-735c-4a1f-915b-582194811e7b admin admin] PATH_INFO: `/v3/projects/ef3a25dbbba2469f953951a11261f297` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:03.014781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381660e-735c-4a1f-915b-582194811e7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef3a25dbbba2469f953951a11261f297)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:03.020943 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381660e-735c-4a1f-915b-582194811e7b admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:03.182111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c381660e-735c-4a1f-915b-582194811e7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef3a25dbbba2469f953951a11261f297', 'request_id': 'req-c381660e-735c-4a1f-915b-582194811e7b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:03.190417 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c381660e-735c-4a1f-915b-582194811e7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef3a25dbbba2469f953951a11261f297', 'request_id': 'req-c381660e-735c-4a1f-915b-582194811e7b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:03.279094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c381660e-735c-4a1f-915b-582194811e7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef3a25dbbba2469f953951a11261f297', 'request_id': 'req-c381660e-735c-4a1f-915b-582194811e7b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:03.311064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c381660e-735c-4a1f-915b-582194811e7b admin admin] The token cache is being invalidate because project ef3a25dbbba2469f953951a11261f297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:15:03.319096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c381660e-735c-4a1f-915b-582194811e7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:03.319096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c381660e-735c-4a1f-915b-582194811e7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:03.319096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1021/2030] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:15:02 2022] DELETE /identity/v3/projects/ef3a25dbbba2469f953951a11261f297 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:03.326063 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47ab42-cd0a-4efa-aa45-cb77488302a2 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:03.326713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47ab42-cd0a-4efa-aa45-cb77488302a2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:03.326713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47ab42-cd0a-4efa-aa45-cb77488302a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:03.359491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-1c47ab42-cd0a-4efa-aa45-cb77488302a2 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:03.361239 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-1c47ab42-cd0a-4efa-aa45-cb77488302a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:03.369058 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:03.464855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1010/2031] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:03.474702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854296c-2cd8-4606-aaa5-4032da7e6e66 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:03.475696 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854296c-2cd8-4606-aaa5-4032da7e6e66 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:03.475982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854296c-2cd8-4606-aaa5-4032da7e6e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:03.500379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-b854296c-2cd8-4606-aaa5-4032da7e6e66 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:03.501582 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b854296c-2cd8-4606-aaa5-4032da7e6e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:03.514885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1022/2032] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:03.520343 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47ab42-cd0a-4efa-aa45-cb77488302a2 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:03.535135 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47ab42-cd0a-4efa-aa45-cb77488302a2 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:03.542695 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1c1d6-2002-4dc1-aadd-fb3ac92ead71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:03.543819 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1c1d6-2002-4dc1-aadd-fb3ac92ead71 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:03.544448 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1c1d6-2002-4dc1-aadd-fb3ac92ead71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:03.545088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1c1d6-2002-4dc1-aadd-fb3ac92ead71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:03.576602 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1c1d6-2002-4dc1-aadd-fb3ac92ead71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:15:03.577783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1c1d6-2002-4dc1-aadd-fb3ac92ead71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:03.587701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1c1d6-2002-4dc1-aadd-fb3ac92ead71 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:03.620452 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1011/2033] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:15:03.624370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854296c-2cd8-4606-aaa5-4032da7e6e66 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:03.642953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854296c-2cd8-4606-aaa5-4032da7e6e66 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:03.666088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4deb3-e03c-4c66-8262-fa4ae38bf5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:03.667033 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4deb3-e03c-4c66-8262-fa4ae38bf5ff admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:03.667416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4deb3-e03c-4c66-8262-fa4ae38bf5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:03.667707 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4deb3-e03c-4c66-8262-fa4ae38bf5ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:03.668441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4deb3-e03c-4c66-8262-fa4ae38bf5ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:03.670471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4deb3-e03c-4c66-8262-fa4ae38bf5ff admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:03.799160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1023/2034] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:15:03 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:03.804310 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1c1d6-2002-4dc1-aadd-fb3ac92ead71 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:03.822846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1c1d6-2002-4dc1-aadd-fb3ac92ead71 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:04.001446 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b54f79-8801-46df-971c-bc52070df310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:04.002230 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b54f79-8801-46df-971c-bc52070df310 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:04.002525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b54f79-8801-46df-971c-bc52070df310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:04.003040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b54f79-8801-46df-971c-bc52070df310 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:04.008106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b54f79-8801-46df-971c-bc52070df310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:04.008106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b54f79-8801-46df-971c-bc52070df310 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:04.124794 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b4c35-1da4-4ae7-9d5d-17ce73632fbb None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:04.125718 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b4c35-1da4-4ae7-9d5d-17ce73632fbb None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:04.126146 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b4c35-1da4-4ae7-9d5d-17ce73632fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:04.159996 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-c88b4c35-1da4-4ae7-9d5d-17ce73632fbb None None] MFA Rules not processed for user `45b1df340f8a4102adf5f680d7e64139`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:04.161880 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c88b4c35-1da4-4ae7-9d5d-17ce73632fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:04.183448 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1012/2035] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:15:03 2022] POST /identity/v3/users => generated 289 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:15:04.185535 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b54f79-8801-46df-971c-bc52070df310 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:04.200026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b54f79-8801-46df-971c-bc52070df310 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:04.206511 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85737e3c-2b30-4386-bcdc-5fb33d0e6dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:04.206511 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85737e3c-2b30-4386-bcdc-5fb33d0e6dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:04.206772 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85737e3c-2b30-4386-bcdc-5fb33d0e6dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:04.206855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85737e3c-2b30-4386-bcdc-5fb33d0e6dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:04.207594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85737e3c-2b30-4386-bcdc-5fb33d0e6dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:04.209976 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85737e3c-2b30-4386-bcdc-5fb33d0e6dd9 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:04.212880 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:04.231488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1013/2036] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:04.239207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85737e3c-2b30-4386-bcdc-5fb33d0e6dd9 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:04.258916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85737e3c-2b30-4386-bcdc-5fb33d0e6dd9 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:04.265674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8ece2-5a27-438f-9557-4749b7806465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:04.266472 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8ece2-5a27-438f-9557-4749b7806465 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:04.266818 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8ece2-5a27-438f-9557-4749b7806465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:04.267105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8ece2-5a27-438f-9557-4749b7806465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:04.267862 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8ece2-5a27-438f-9557-4749b7806465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:04.269563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8ece2-5a27-438f-9557-4749b7806465 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:04.285810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1014/2037] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:04.290657 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8ece2-5a27-438f-9557-4749b7806465 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:04.314485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8ece2-5a27-438f-9557-4749b7806465 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:04.315492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1024/2038] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:15:04.320745 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b4c35-1da4-4ae7-9d5d-17ce73632fbb None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:04.324263 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c88b4c35-1da4-4ae7-9d5d-17ce73632fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:04.332329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412e31d-8573-45e2-b59d-c117e1db1eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:04.333270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412e31d-8573-45e2-b59d-c117e1db1eb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:04.333564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412e31d-8573-45e2-b59d-c117e1db1eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:04.333884 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412e31d-8573-45e2-b59d-c117e1db1eb1 admin admin] PATH_INFO: `/v3/projects/5134df78edfd4a13a7b781db30344d9d/users/9e7007a0ee8b4633a36efa32d10f46e4/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:04.359193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b4c35-1da4-4ae7-9d5d-17ce73632fbb None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:04.368090 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe7f51-7fb4-4ff0-9065-0e16cfce05da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:04.368389 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412e31d-8573-45e2-b59d-c117e1db1eb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5134df78edfd4a13a7b781db30344d9d, user_id=9e7007a0ee8b4633a36efa32d10f46e4, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:04.368972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe7f51-7fb4-4ff0-9065-0e16cfce05da admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:04.372868 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe7f51-7fb4-4ff0-9065-0e16cfce05da admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:04.372868 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe7f51-7fb4-4ff0-9065-0e16cfce05da admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:04.372868 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe7f51-7fb4-4ff0-9065-0e16cfce05da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:04.373440 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412e31d-8573-45e2-b59d-c117e1db1eb1 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:04.373526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe7f51-7fb4-4ff0-9065-0e16cfce05da admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:04.519301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1015/2039] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:04 2022] PUT /identity/v3/projects/5134df78edfd4a13a7b781db30344d9d/users/9e7007a0ee8b4633a36efa32d10f46e4/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:04.519599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1025/2040] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:15:04 2022] POST /identity/v3/projects => generated 383 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:04.523896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412e31d-8573-45e2-b59d-c117e1db1eb1 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:04.537563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879bdbb-6b07-4af4-9623-bed80ac54539 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:04.538240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879bdbb-6b07-4af4-9623-bed80ac54539 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:04.538802 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879bdbb-6b07-4af4-9623-bed80ac54539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:04.595767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-4879bdbb-6b07-4af4-9623-bed80ac54539 None None] MFA Rules not processed for user `9e7007a0ee8b4633a36efa32d10f46e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:04.597310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4879bdbb-6b07-4af4-9623-bed80ac54539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:04.613098 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:04.643426 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412e31d-8573-45e2-b59d-c117e1db1eb1 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:04.761968 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db028d7-81e0-470f-93b9-0d7650d2086a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:04.763892 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db028d7-81e0-470f-93b9-0d7650d2086a admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:04.764315 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db028d7-81e0-470f-93b9-0d7650d2086a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:04.764670 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db028d7-81e0-470f-93b9-0d7650d2086a admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:04.765719 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db028d7-81e0-470f-93b9-0d7650d2086a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:04.768109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db028d7-81e0-470f-93b9-0d7650d2086a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:04.769823 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1026/2041] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:04.783898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879bdbb-6b07-4af4-9623-bed80ac54539 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:04.786098 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4879bdbb-6b07-4af4-9623-bed80ac54539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:04.875806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1016/2042] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:15:04 2022] POST /identity/v3/users => generated 298 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:04.879918 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db028d7-81e0-470f-93b9-0d7650d2086a admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:04.899480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db028d7-81e0-470f-93b9-0d7650d2086a admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:04.940145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5fde1-c342-442e-83aa-ab29b61c0f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:04.942520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5fde1-c342-442e-83aa-ab29b61c0f01 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:04.948134 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5fde1-c342-442e-83aa-ab29b61c0f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:04.958790 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5fde1-c342-442e-83aa-ab29b61c0f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:04.958790 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5fde1-c342-442e-83aa-ab29b61c0f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:04.958790 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5fde1-c342-442e-83aa-ab29b61c0f01 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:04.977590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1017/2043] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:04.987635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5fde1-c342-442e-83aa-ab29b61c0f01 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:05.011855 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879bdbb-6b07-4af4-9623-bed80ac54539 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:05.025096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5fde1-c342-442e-83aa-ab29b61c0f01 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:05.026076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b35b48-a2f6-4bea-9d48-ca3265f8265c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8dcb0a0f2c4a2d955544324213739c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:05.026983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b35b48-a2f6-4bea-9d48-ca3265f8265c service glance] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:05.027362 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b35b48-a2f6-4bea-9d48-ca3265f8265c service glance] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:05.027716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b35b48-a2f6-4bea-9d48-ca3265f8265c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:05.042112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4612f44-7660-4f41-8b0c-7e102a90be0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:05.044987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4612f44-7660-4f41-8b0c-7e102a90be0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:05.045345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4612f44-7660-4f41-8b0c-7e102a90be0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:05.045609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4612f44-7660-4f41-8b0c-7e102a90be0d admin admin] PATH_INFO: `/v3/projects/608fd13d890e4744b9c6213232b47433/users/ac71e333fa5d40b0b0b8d8fe77d3c68b/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:05.055409 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b35b48-a2f6-4bea-9d48-ca3265f8265c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:05.057660 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b35b48-a2f6-4bea-9d48-ca3265f8265c service glance] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:05.081716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1027/2044] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:05.098453 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4612f44-7660-4f41-8b0c-7e102a90be0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=608fd13d890e4744b9c6213232b47433, user_id=ac71e333fa5d40b0b0b8d8fe77d3c68b, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:05.103169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4612f44-7660-4f41-8b0c-7e102a90be0d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:05.228032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1018/2045] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:04 2022] PUT /identity/v3/projects/608fd13d890e4744b9c6213232b47433/users/ac71e333fa5d40b0b0b8d8fe77d3c68b/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:05.229426 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b35b48-a2f6-4bea-9d48-ca3265f8265c service glance] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:05.268111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b35b48-a2f6-4bea-9d48-ca3265f8265c service glance] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:05.406760 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:05.408662 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09784224-9c20-4699-adb6-1e26c91d3add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:05.410118 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09784224-9c20-4699-adb6-1e26c91d3add admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:05.410555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09784224-9c20-4699-adb6-1e26c91d3add admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:05.410873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09784224-9c20-4699-adb6-1e26c91d3add admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:05.411607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09784224-9c20-4699-adb6-1e26c91d3add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:05.415677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09784224-9c20-4699-adb6-1e26c91d3add admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:05.432645 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1028/2046] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:15:05.454933 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4612f44-7660-4f41-8b0c-7e102a90be0d admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:05.461316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4612f44-7660-4f41-8b0c-7e102a90be0d admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:05.486132 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3e225-8baf-4f17-81a6-ff4d0bc0e1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:05.497482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3e225-8baf-4f17-81a6-ff4d0bc0e1f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:05.497482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3e225-8baf-4f17-81a6-ff4d0bc0e1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:05.497482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3e225-8baf-4f17-81a6-ff4d0bc0e1f2 admin admin] PATH_INFO: `/v3/projects/608fd13d890e4744b9c6213232b47433/users/ac71e333fa5d40b0b0b8d8fe77d3c68b/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:05.498235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3e225-8baf-4f17-81a6-ff4d0bc0e1f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=608fd13d890e4744b9c6213232b47433, user_id=ac71e333fa5d40b0b0b8d8fe77d3c68b, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:05.502035 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3e225-8baf-4f17-81a6-ff4d0bc0e1f2 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:05.531410 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1019/2047] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:05 2022] PUT /identity/v3/projects/608fd13d890e4744b9c6213232b47433/users/ac71e333fa5d40b0b0b8d8fe77d3c68b/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:14.415391 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09784224-9c20-4699-adb6-1e26c91d3add admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:14.418966 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-09784224-9c20-4699-adb6-1e26c91d3add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:14.455281 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09784224-9c20-4699-adb6-1e26c91d3add admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:14.666699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f077325-c3f3-45da-8d01-74702148e074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:14.668411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f077325-c3f3-45da-8d01-74702148e074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:14.669264 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f077325-c3f3-45da-8d01-74702148e074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:14.669777 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f077325-c3f3-45da-8d01-74702148e074 admin admin] PATH_INFO: `/v3/users/21de90974572419e906e7ac5d9d8292d` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:14.671475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f077325-c3f3-45da-8d01-74702148e074 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21de90974572419e906e7ac5d9d8292d)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:14.673730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f077325-c3f3-45da-8d01-74702148e074 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:15.088969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6f077325-c3f3-45da-8d01-74702148e074 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21de90974572419e906e7ac5d9d8292d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:15.104520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6f077325-c3f3-45da-8d01-74702148e074 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21de90974572419e906e7ac5d9d8292d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:15.106029 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6f077325-c3f3-45da-8d01-74702148e074 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21de90974572419e906e7ac5d9d8292d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:15.138860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6f077325-c3f3-45da-8d01-74702148e074 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21de90974572419e906e7ac5d9d8292d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:15.243823 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6f077325-c3f3-45da-8d01-74702148e074 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21de90974572419e906e7ac5d9d8292d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:15.262315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1029/2048] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:15:14 2022] DELETE /identity/v3/users/21de90974572419e906e7ac5d9d8292d => generated 0 bytes in 848 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:15.525131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=55,keystone:INSERT=5 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:15.652489 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3e225-8baf-4f17-81a6-ff4d0bc0e1f2 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:15.657655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b9c3e225-8baf-4f17-81a6-ff4d0bc0e1f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:15.712827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3e225-8baf-4f17-81a6-ff4d0bc0e1f2 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:15.813248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53de80-97d4-4ebe-9f50-84523671eeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:15.814328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53de80-97d4-4ebe-9f50-84523671eeb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:15.814655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53de80-97d4-4ebe-9f50-84523671eeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:15.814942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53de80-97d4-4ebe-9f50-84523671eeb8 admin admin] PATH_INFO: `/v3/projects/9a337533276641a2b556a55fdab4a049` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:15.825151 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53de80-97d4-4ebe-9f50-84523671eeb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a337533276641a2b556a55fdab4a049)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:15.825151 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53de80-97d4-4ebe-9f50-84523671eeb8 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:16.089149 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cb53de80-97d4-4ebe-9f50-84523671eeb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a337533276641a2b556a55fdab4a049', 'request_id': 'req-cb53de80-97d4-4ebe-9f50-84523671eeb8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:16.102722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cb53de80-97d4-4ebe-9f50-84523671eeb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a337533276641a2b556a55fdab4a049', 'request_id': 'req-cb53de80-97d4-4ebe-9f50-84523671eeb8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:16.112682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cb53de80-97d4-4ebe-9f50-84523671eeb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a337533276641a2b556a55fdab4a049', 'request_id': 'req-cb53de80-97d4-4ebe-9f50-84523671eeb8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:16.169030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f077325-c3f3-45da-8d01-74702148e074 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:16.171530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-6f077325-c3f3-45da-8d01-74702148e074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:16.197828 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f077325-c3f3-45da-8d01-74702148e074 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:16.267476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:16.289509 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cb53de80-97d4-4ebe-9f50-84523671eeb8 admin admin] The token cache is being invalidate because project 9a337533276641a2b556a55fdab4a049 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:15:16.294041 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cb53de80-97d4-4ebe-9f50-84523671eeb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:16.295848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cb53de80-97d4-4ebe-9f50-84523671eeb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:16.302595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1020/2049] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:15:15 2022] DELETE /identity/v3/projects/9a337533276641a2b556a55fdab4a049 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:16.311059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993230c-4cf9-4703-a11e-2a060f2af947 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:16.311690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993230c-4cf9-4703-a11e-2a060f2af947 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:16.312158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993230c-4cf9-4703-a11e-2a060f2af947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:16.369635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-7993230c-4cf9-4703-a11e-2a060f2af947 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:16.371292 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7993230c-4cf9-4703-a11e-2a060f2af947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:16.405780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ebaed-3a29-485c-a0b6-8a19859b3cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:16.406911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ebaed-3a29-485c-a0b6-8a19859b3cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:16.407059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ebaed-3a29-485c-a0b6-8a19859b3cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:16.407516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ebaed-3a29-485c-a0b6-8a19859b3cab admin admin] PATH_INFO: `/v3/users/3891ff1209124d92864a9230b1229feb` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:16.409129 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ebaed-3a29-485c-a0b6-8a19859b3cab admin admin] RBAC: Authorizing `identity:delete_user(user_id=3891ff1209124d92864a9230b1229feb)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:16.411935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ebaed-3a29-485c-a0b6-8a19859b3cab admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:16.517655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1021/2050] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:16.536706 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc188c-10a3-4b20-96fb-8978ed601df1 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:16.539947 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc188c-10a3-4b20-96fb-8978ed601df1 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:16.540416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc188c-10a3-4b20-96fb-8978ed601df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:16.612051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-e0cc188c-10a3-4b20-96fb-8978ed601df1 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:16.617250 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e0cc188c-10a3-4b20-96fb-8978ed601df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:16.656185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1022/2051] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:16.657641 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc188c-10a3-4b20-96fb-8978ed601df1 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:16.685007 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc188c-10a3-4b20-96fb-8978ed601df1 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:16.694829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a25cc3-b3cf-4e3a-8f38-ca61b30aba8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:16.696341 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a25cc3-b3cf-4e3a-8f38-ca61b30aba8a admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:16.697430 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a25cc3-b3cf-4e3a-8f38-ca61b30aba8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:16.698052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a25cc3-b3cf-4e3a-8f38-ca61b30aba8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:16.699268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a25cc3-b3cf-4e3a-8f38-ca61b30aba8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:15:16.699767 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a25cc3-b3cf-4e3a-8f38-ca61b30aba8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:16.701619 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a25cc3-b3cf-4e3a-8f38-ca61b30aba8a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:16.743730 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1023/2052] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:15:16.744778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a25cc3-b3cf-4e3a-8f38-ca61b30aba8a admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:16.775191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a25cc3-b3cf-4e3a-8f38-ca61b30aba8a admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:16.784676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-910ebaed-3a29-485c-a0b6-8a19859b3cab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3891ff1209124d92864a9230b1229feb'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:16.792056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d94999-60a4-4794-a20e-d2150d9c34f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:16.792756 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d94999-60a4-4794-a20e-d2150d9c34f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:16.793044 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d94999-60a4-4794-a20e-d2150d9c34f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:16.793324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d94999-60a4-4794-a20e-d2150d9c34f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:16.794043 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d94999-60a4-4794-a20e-d2150d9c34f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:16.796162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d94999-60a4-4794-a20e-d2150d9c34f4 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:16.811205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:16.814714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-910ebaed-3a29-485c-a0b6-8a19859b3cab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3891ff1209124d92864a9230b1229feb'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:16.817175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-910ebaed-3a29-485c-a0b6-8a19859b3cab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3891ff1209124d92864a9230b1229feb'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:16.838250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-910ebaed-3a29-485c-a0b6-8a19859b3cab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3891ff1209124d92864a9230b1229feb'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:16.929213 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-910ebaed-3a29-485c-a0b6-8a19859b3cab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3891ff1209124d92864a9230b1229feb'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:16.971806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1024/2053] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:15:16 2022] POST /identity/v3/projects => generated 383 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:16.972275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1030/2054] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:15:16 2022] DELETE /identity/v3/users/3891ff1209124d92864a9230b1229feb => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:16.979348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ebaed-3a29-485c-a0b6-8a19859b3cab admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:16.981190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a24fe-346c-4c7e-8522-096f15876b97 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:16.982406 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a24fe-346c-4c7e-8522-096f15876b97 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:16.982898 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a24fe-346c-4c7e-8522-096f15876b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:16.983524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-910ebaed-3a29-485c-a0b6-8a19859b3cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:17.060338 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-8e4a24fe-346c-4c7e-8522-096f15876b97 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:17.061744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-8e4a24fe-346c-4c7e-8522-096f15876b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:17.130375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ebaed-3a29-485c-a0b6-8a19859b3cab admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:17.206429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1025/2055] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:17.215894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a24fe-346c-4c7e-8522-096f15876b97 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:17.218593 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-8e4a24fe-346c-4c7e-8522-096f15876b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:17.290282 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:17.315034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a24fe-346c-4c7e-8522-096f15876b97 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:17.316528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d136646-1e3b-4f06-b662-2d16a7ac27c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:17.317361 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d136646-1e3b-4f06-b662-2d16a7ac27c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:17.317766 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d136646-1e3b-4f06-b662-2d16a7ac27c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:17.320100 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d136646-1e3b-4f06-b662-2d16a7ac27c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:17.320630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d136646-1e3b-4f06-b662-2d16a7ac27c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:17.322805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d136646-1e3b-4f06-b662-2d16a7ac27c5 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:17.334102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a76e95-908e-4b81-9845-4e2cd15fefab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:17.340175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a76e95-908e-4b81-9845-4e2cd15fefab service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:17.340693 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a76e95-908e-4b81-9845-4e2cd15fefab service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:17.341167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a76e95-908e-4b81-9845-4e2cd15fefab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:17.373226 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a76e95-908e-4b81-9845-4e2cd15fefab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:17.385261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a76e95-908e-4b81-9845-4e2cd15fefab service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:17.449815 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1026/2056] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:17.555167 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1031/2057] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:15:16 2022] POST /identity/v3/users => generated 292 bytes in 576 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:17.559784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a76e95-908e-4b81-9845-4e2cd15fefab service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:17.573699 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a76e95-908e-4b81-9845-4e2cd15fefab service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:17.580670 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5e633-78b5-4487-8a9b-2d385fd5791d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:17.581851 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5e633-78b5-4487-8a9b-2d385fd5791d admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:17.582261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5e633-78b5-4487-8a9b-2d385fd5791d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:17.582435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5e633-78b5-4487-8a9b-2d385fd5791d admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:17.583310 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5e633-78b5-4487-8a9b-2d385fd5791d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:17.586479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5e633-78b5-4487-8a9b-2d385fd5791d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:17.601555 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1027/2058] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:17.607717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d136646-1e3b-4f06-b662-2d16a7ac27c5 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:17.633587 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d136646-1e3b-4f06-b662-2d16a7ac27c5 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:17.642588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb04424-adb8-4eba-a4d5-aca2f2297840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:17.643386 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb04424-adb8-4eba-a4d5-aca2f2297840 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:17.643810 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb04424-adb8-4eba-a4d5-aca2f2297840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:17.644247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb04424-adb8-4eba-a4d5-aca2f2297840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:17.645177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb04424-adb8-4eba-a4d5-aca2f2297840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:17.647826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb04424-adb8-4eba-a4d5-aca2f2297840 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:17.667330 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1032/2059] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:17.669753 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5e633-78b5-4487-8a9b-2d385fd5791d admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:17.694586 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5e633-78b5-4487-8a9b-2d385fd5791d admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:17.715620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5982c-6265-4006-b17c-94f3d26e5e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:17.721127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5982c-6265-4006-b17c-94f3d26e5e71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:17.721127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5982c-6265-4006-b17c-94f3d26e5e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:17.721127 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5982c-6265-4006-b17c-94f3d26e5e71 admin admin] PATH_INFO: `/v3/projects/3087891c94f34fc2a6234f04c03f27e0/users/6c85cf89ac304a9caeac72b5805d0dfc/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:17.745765 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5982c-6265-4006-b17c-94f3d26e5e71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3087891c94f34fc2a6234f04c03f27e0, user_id=6c85cf89ac304a9caeac72b5805d0dfc, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:17.750973 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5982c-6265-4006-b17c-94f3d26e5e71 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:17.900958 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f8a42-a9da-41e7-b933-e4e1bef01aa8 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:17.901336 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f8a42-a9da-41e7-b933-e4e1bef01aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:17.901757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f8a42-a9da-41e7-b933-e4e1bef01aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:17.906857 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1028/2060] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:17 2022] PUT /identity/v3/projects/3087891c94f34fc2a6234f04c03f27e0/users/6c85cf89ac304a9caeac72b5805d0dfc/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:17.959924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-c49f8a42-a9da-41e7-b933-e4e1bef01aa8 None None] MFA Rules not processed for user `6c85cf89ac304a9caeac72b5805d0dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:17.961710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c49f8a42-a9da-41e7-b933-e4e1bef01aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:18.061112 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1033/2061] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:18.073207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5982c-6265-4006-b17c-94f3d26e5e71 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:18.075631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a3c5982c-6265-4006-b17c-94f3d26e5e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:18.159047 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:18.186071 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946bccb-72c7-4ee9-afc2-7a6354bc039c None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:18.186071 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946bccb-72c7-4ee9-afc2-7a6354bc039c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:18.186071 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946bccb-72c7-4ee9-afc2-7a6354bc039c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:18.193001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5982c-6265-4006-b17c-94f3d26e5e71 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:18.200716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd565ac-d028-4336-90ac-6f2a670b3f2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:18.203032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd565ac-d028-4336-90ac-6f2a670b3f2b service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:18.203504 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd565ac-d028-4336-90ac-6f2a670b3f2b service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:18.208757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd565ac-d028-4336-90ac-6f2a670b3f2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:18.219396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd565ac-d028-4336-90ac-6f2a670b3f2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:18.220466 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd565ac-d028-4336-90ac-6f2a670b3f2b service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:18.232901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1029/2062] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:18.234911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd565ac-d028-4336-90ac-6f2a670b3f2b service nova] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:18.236446 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6cd565ac-d028-4336-90ac-6f2a670b3f2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:18.244780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-b946bccb-72c7-4ee9-afc2-7a6354bc039c None None] MFA Rules not processed for user `ac71e333fa5d40b0b0b8d8fe77d3c68b`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:18.246041 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b946bccb-72c7-4ee9-afc2-7a6354bc039c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:18.345487 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd565ac-d028-4336-90ac-6f2a670b3f2b service nova] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:18.352120 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5de00-6d69-491e-b229-3a35daa63ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:18.353021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5de00-6d69-491e-b229-3a35daa63ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:18.353318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5de00-6d69-491e-b229-3a35daa63ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:18.353606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5de00-6d69-491e-b229-3a35daa63ae7 admin admin] PATH_INFO: `/v3/projects/d9d88964fca14a2e978710772e23a598` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:18.354972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5de00-6d69-491e-b229-3a35daa63ae7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9d88964fca14a2e978710772e23a598)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:18.357040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5de00-6d69-491e-b229-3a35daa63ae7 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:18.381511 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1034/2063] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:18.390087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946bccb-72c7-4ee9-afc2-7a6354bc039c None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:18.392009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b946bccb-72c7-4ee9-afc2-7a6354bc039c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:18.461654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946bccb-72c7-4ee9-afc2-7a6354bc039c None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:18.466719 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4c8b0-f909-4917-8bc5-a087f6ca4be7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8dcb0a0f2c4a2d955544324213739c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:18.466719 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4c8b0-f909-4917-8bc5-a087f6ca4be7 service glance] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:18.467193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4c8b0-f909-4917-8bc5-a087f6ca4be7 service glance] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:18.467193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4c8b0-f909-4917-8bc5-a087f6ca4be7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:18.470496 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-00d4c8b0-f909-4917-8bc5-a087f6ca4be7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:18.482136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0bd5de00-6d69-491e-b229-3a35daa63ae7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9d88964fca14a2e978710772e23a598', 'request_id': 'req-0bd5de00-6d69-491e-b229-3a35daa63ae7'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:18.491964 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0bd5de00-6d69-491e-b229-3a35daa63ae7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9d88964fca14a2e978710772e23a598', 'request_id': 'req-0bd5de00-6d69-491e-b229-3a35daa63ae7'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:18.501275 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0bd5de00-6d69-491e-b229-3a35daa63ae7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9d88964fca14a2e978710772e23a598', 'request_id': 'req-0bd5de00-6d69-491e-b229-3a35daa63ae7'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:18.577520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4c8b0-f909-4917-8bc5-a087f6ca4be7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:18.584723 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4c8b0-f909-4917-8bc5-a087f6ca4be7 service glance] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:18.611949 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1035/2064] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:15:18.635826 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0bd5de00-6d69-491e-b229-3a35daa63ae7 admin admin] The token cache is being invalidate because project d9d88964fca14a2e978710772e23a598 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:15:18.638218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0bd5de00-6d69-491e-b229-3a35daa63ae7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:18.639963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0bd5de00-6d69-491e-b229-3a35daa63ae7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:18.649201 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1030/2065] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:15:18 2022] DELETE /identity/v3/projects/d9d88964fca14a2e978710772e23a598 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:18.655797 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4c8b0-f909-4917-8bc5-a087f6ca4be7 service glance] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:18.682232 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4c8b0-f909-4917-8bc5-a087f6ca4be7 service glance] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:18.867692 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb151e1c-7a83-4b6b-887b-603409894e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:18.873418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151e1c-7a83-4b6b-887b-603409894e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:18.875974 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151e1c-7a83-4b6b-887b-603409894e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:18.878013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151e1c-7a83-4b6b-887b-603409894e82 admin admin] PATH_INFO: `/v3/users/7649e853b6424f47808de5ccc6aa4624` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:18.882776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb151e1c-7a83-4b6b-887b-603409894e82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7649e853b6424f47808de5ccc6aa4624)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:18.891467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb151e1c-7a83-4b6b-887b-603409894e82 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:19.200905 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb969019-60af-46df-a65b-6d2ffc72f0d5 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:19.201752 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb969019-60af-46df-a65b-6d2ffc72f0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:19.202499 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb969019-60af-46df-a65b-6d2ffc72f0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:19.233683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-eb969019-60af-46df-a65b-6d2ffc72f0d5 None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:19.235392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-eb969019-60af-46df-a65b-6d2ffc72f0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:19.344239 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1031/2066] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Jul 18 15:15:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:19.402277 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-cb151e1c-7a83-4b6b-887b-603409894e82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7649e853b6424f47808de5ccc6aa4624'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:19.427383 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-cb151e1c-7a83-4b6b-887b-603409894e82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7649e853b6424f47808de5ccc6aa4624'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:19.431822 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-cb151e1c-7a83-4b6b-887b-603409894e82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7649e853b6424f47808de5ccc6aa4624'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:19.457257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb969019-60af-46df-a65b-6d2ffc72f0d5 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:19.457257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-eb969019-60af-46df-a65b-6d2ffc72f0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:19.476240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-cb151e1c-7a83-4b6b-887b-603409894e82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7649e853b6424f47808de5ccc6aa4624'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:19.645254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-cb151e1c-7a83-4b6b-887b-603409894e82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7649e853b6424f47808de5ccc6aa4624'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:19.655684 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1036/2067] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:15:18 2022] DELETE /identity/v3/users/7649e853b6424f47808de5ccc6aa4624 => generated 0 bytes in 1001 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:15:19.669633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb969019-60af-46df-a65b-6d2ffc72f0d5 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:19.769594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a1f2e-165d-471d-bf65-830c31be7c2b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:19.777345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a1f2e-165d-471d-bf65-830c31be7c2b service swift] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:19.777856 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a1f2e-165d-471d-bf65-830c31be7c2b service swift] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:19.778157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a1f2e-165d-471d-bf65-830c31be7c2b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:19.780891 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-fc1a1f2e-165d-471d-bf65-830c31be7c2b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:19.782977 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:19.861464 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a1f2e-165d-471d-bf65-830c31be7c2b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:19.863275 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a1f2e-165d-471d-bf65-830c31be7c2b service swift] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:19.874749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1032/2068] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:15:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 15:15:20.057825 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:21.041093 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb151e1c-7a83-4b6b-887b-603409894e82 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:21.044422 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-cb151e1c-7a83-4b6b-887b-603409894e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:21.255736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:21.283991 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb151e1c-7a83-4b6b-887b-603409894e82 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:21.291598 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72595c11-a190-4da5-8b36-4ae0f79f5cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:21.300288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72595c11-a190-4da5-8b36-4ae0f79f5cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:21.300288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72595c11-a190-4da5-8b36-4ae0f79f5cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:21.300288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72595c11-a190-4da5-8b36-4ae0f79f5cb6 admin admin] PATH_INFO: `/v3/projects/a075e285054647a79931ba1354224f05` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:21.300288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72595c11-a190-4da5-8b36-4ae0f79f5cb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a075e285054647a79931ba1354224f05)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:21.300288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72595c11-a190-4da5-8b36-4ae0f79f5cb6 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:21.555056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-72595c11-a190-4da5-8b36-4ae0f79f5cb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a075e285054647a79931ba1354224f05', 'request_id': 'req-72595c11-a190-4da5-8b36-4ae0f79f5cb6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:21.571976 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-72595c11-a190-4da5-8b36-4ae0f79f5cb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a075e285054647a79931ba1354224f05', 'request_id': 'req-72595c11-a190-4da5-8b36-4ae0f79f5cb6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:21.771158 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-72595c11-a190-4da5-8b36-4ae0f79f5cb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a075e285054647a79931ba1354224f05', 'request_id': 'req-72595c11-a190-4da5-8b36-4ae0f79f5cb6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:21.826464 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-72595c11-a190-4da5-8b36-4ae0f79f5cb6 admin admin] The token cache is being invalidate because project a075e285054647a79931ba1354224f05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:15:21.829539 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-72595c11-a190-4da5-8b36-4ae0f79f5cb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:21.832548 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-72595c11-a190-4da5-8b36-4ae0f79f5cb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:21.842172 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1037/2069] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:15:21 2022] DELETE /identity/v3/projects/a075e285054647a79931ba1354224f05 => generated 0 bytes in 802 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:21.847185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7ccda-90b8-4bc7-aa31-f3a2eb964437 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:21.848171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7ccda-90b8-4bc7-aa31-f3a2eb964437 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:21.848620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7ccda-90b8-4bc7-aa31-f3a2eb964437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:21.904215 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-7dd7ccda-90b8-4bc7-aa31-f3a2eb964437 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:21.905534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7dd7ccda-90b8-4bc7-aa31-f3a2eb964437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:22.047579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1033/2070] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:22.056559 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec88fa-4101-4fba-bdd2-8def2046658f None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:22.057162 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec88fa-4101-4fba-bdd2-8def2046658f None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:22.057162 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec88fa-4101-4fba-bdd2-8def2046658f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:22.111362 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-d9ec88fa-4101-4fba-bdd2-8def2046658f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:22.120080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d9ec88fa-4101-4fba-bdd2-8def2046658f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:22.144775 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1038/2071] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:22.149429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7ccda-90b8-4bc7-aa31-f3a2eb964437 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:22.166793 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7ccda-90b8-4bc7-aa31-f3a2eb964437 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:22.199402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76504daa-1d61-4691-9cb3-4744dd06da43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:22.216621 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76504daa-1d61-4691-9cb3-4744dd06da43 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:22.217218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76504daa-1d61-4691-9cb3-4744dd06da43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:22.217667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76504daa-1d61-4691-9cb3-4744dd06da43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:22.223048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76504daa-1d61-4691-9cb3-4744dd06da43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:15:22.224939 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76504daa-1d61-4691-9cb3-4744dd06da43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:22.227466 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76504daa-1d61-4691-9cb3-4744dd06da43 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:22.251088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1034/2072] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:15:22.257919 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec88fa-4101-4fba-bdd2-8def2046658f None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:22.282320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec88fa-4101-4fba-bdd2-8def2046658f None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:22.306586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9543bb-164d-4721-bee6-69ef22822940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:22.308630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9543bb-164d-4721-bee6-69ef22822940 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:22.309849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9543bb-164d-4721-bee6-69ef22822940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:22.310225 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9543bb-164d-4721-bee6-69ef22822940 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:22.319569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:22.322147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9543bb-164d-4721-bee6-69ef22822940 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:22.326917 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9543bb-164d-4721-bee6-69ef22822940 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:22.450398 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1039/2073] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:15:22 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:22.455884 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76504daa-1d61-4691-9cb3-4744dd06da43 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:22.473770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76504daa-1d61-4691-9cb3-4744dd06da43 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:22.743931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612288e-d1a1-40d4-a74b-9b6ed08aa762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:22.744585 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612288e-d1a1-40d4-a74b-9b6ed08aa762 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:22.744984 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612288e-d1a1-40d4-a74b-9b6ed08aa762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:22.745375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612288e-d1a1-40d4-a74b-9b6ed08aa762 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:22.746230 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d612288e-d1a1-40d4-a74b-9b6ed08aa762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:22.751056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d612288e-d1a1-40d4-a74b-9b6ed08aa762 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:22.941048 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9543bb-164d-4721-bee6-69ef22822940 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:22.942451 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1035/2074] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:15:22 2022] POST /identity/v3/users => generated 293 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:23.005844 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9543bb-164d-4721-bee6-69ef22822940 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:23.036766 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77170493-88ee-43d9-86ef-2587457e3225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:23.038928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77170493-88ee-43d9-86ef-2587457e3225 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:23.042560 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77170493-88ee-43d9-86ef-2587457e3225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:23.043299 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77170493-88ee-43d9-86ef-2587457e3225 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:23.044860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77170493-88ee-43d9-86ef-2587457e3225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:23.058312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77170493-88ee-43d9-86ef-2587457e3225 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:23.118973 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1040/2075] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:23.125829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612288e-d1a1-40d4-a74b-9b6ed08aa762 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:23.155059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612288e-d1a1-40d4-a74b-9b6ed08aa762 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:23.166510 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5456b3-ec6e-48fe-be31-149ec9668e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:23.170121 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5456b3-ec6e-48fe-be31-149ec9668e5b admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:23.171109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5456b3-ec6e-48fe-be31-149ec9668e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:23.171877 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5456b3-ec6e-48fe-be31-149ec9668e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:23.173940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5456b3-ec6e-48fe-be31-149ec9668e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:23.178760 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5456b3-ec6e-48fe-be31-149ec9668e5b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:23.205726 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1036/2076] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:23.213036 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77170493-88ee-43d9-86ef-2587457e3225 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:23.248918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77170493-88ee-43d9-86ef-2587457e3225 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:23.298995 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d4f98-e1b9-4db0-a7dc-fd15f3397cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:23.307245 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d4f98-e1b9-4db0-a7dc-fd15f3397cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:23.308138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d4f98-e1b9-4db0-a7dc-fd15f3397cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:23.308761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d4f98-e1b9-4db0-a7dc-fd15f3397cdd admin admin] PATH_INFO: `/v3/projects/2fad77febdbf4b5d85253f80e566e89f/users/331e27de24d248a8b97e2f008d890e02/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:23.370810 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d4f98-e1b9-4db0-a7dc-fd15f3397cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fad77febdbf4b5d85253f80e566e89f, user_id=331e27de24d248a8b97e2f008d890e02, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:23.375152 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d4f98-e1b9-4db0-a7dc-fd15f3397cdd admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:23.505114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1041/2077] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:23 2022] PUT /identity/v3/projects/2fad77febdbf4b5d85253f80e566e89f/users/331e27de24d248a8b97e2f008d890e02/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:23.531171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c520686-4348-4ca6-b06d-0f389abd4e88 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:23.532059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c520686-4348-4ca6-b06d-0f389abd4e88 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:23.533374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c520686-4348-4ca6-b06d-0f389abd4e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:23.639126 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-8c520686-4348-4ca6-b06d-0f389abd4e88 None None] MFA Rules not processed for user `331e27de24d248a8b97e2f008d890e02`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:23.642085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-8c520686-4348-4ca6-b06d-0f389abd4e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:23.833261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1037/2078] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:23.835224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d4f98-e1b9-4db0-a7dc-fd15f3397cdd admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:23.905548 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:23.938476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d4f98-e1b9-4db0-a7dc-fd15f3397cdd admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:23.950950 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aaf812-1afb-4521-bb10-e3aa6ec5ebd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:23.951657 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaf812-1afb-4521-bb10-e3aa6ec5ebd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:23.951893 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaf812-1afb-4521-bb10-e3aa6ec5ebd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:23.952376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaf812-1afb-4521-bb10-e3aa6ec5ebd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:23.953394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aaf812-1afb-4521-bb10-e3aa6ec5ebd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:23.956133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aaf812-1afb-4521-bb10-e3aa6ec5ebd0 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:24.146914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1042/2079] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:15:23 2022] POST /identity/v3/projects => generated 387 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:24.152800 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c520686-4348-4ca6-b06d-0f389abd4e88 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:24.167680 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c520686-4348-4ca6-b06d-0f389abd4e88 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:24.262302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:24.268930 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a31b06-2049-4131-866f-56221acd089c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:24.269842 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a31b06-2049-4131-866f-56221acd089c admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:24.270408 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a31b06-2049-4131-866f-56221acd089c admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:24.270731 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a31b06-2049-4131-866f-56221acd089c admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:24.271485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a31b06-2049-4131-866f-56221acd089c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:24.274185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a31b06-2049-4131-866f-56221acd089c admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:24.401111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aaf812-1afb-4521-bb10-e3aa6ec5ebd0 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:24.410651 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1038/2080] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:15:24 2022] POST /identity/v3/users => generated 294 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:24.450326 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aaf812-1afb-4521-bb10-e3aa6ec5ebd0 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:24.466383 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff16a3d-3bc7-4a6c-9a97-8b377c27eb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:24.467754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff16a3d-3bc7-4a6c-9a97-8b377c27eb60 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:24.470402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff16a3d-3bc7-4a6c-9a97-8b377c27eb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:24.472126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff16a3d-3bc7-4a6c-9a97-8b377c27eb60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:24.475709 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff16a3d-3bc7-4a6c-9a97-8b377c27eb60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:24.504917 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff16a3d-3bc7-4a6c-9a97-8b377c27eb60 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:24.580894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a31b06-2049-4131-866f-56221acd089c admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:24.592646 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1043/2081] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:24.634509 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a31b06-2049-4131-866f-56221acd089c admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:24.661339 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedad34-8a32-4697-b114-83af47c294ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:24.663232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedad34-8a32-4697-b114-83af47c294ef admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:24.663766 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedad34-8a32-4697-b114-83af47c294ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:24.664296 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedad34-8a32-4697-b114-83af47c294ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:24.680910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdedad34-8a32-4697-b114-83af47c294ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:24.682707 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdedad34-8a32-4697-b114-83af47c294ef admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:24.696073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1039/2082] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:15:24.700970 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff16a3d-3bc7-4a6c-9a97-8b377c27eb60 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:24.726051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff16a3d-3bc7-4a6c-9a97-8b377c27eb60 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:24.734855 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06bb25-4859-4c77-ac58-931b4204f7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:24.736602 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06bb25-4859-4c77-ac58-931b4204f7be admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:24.736602 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06bb25-4859-4c77-ac58-931b4204f7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:24.736883 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06bb25-4859-4c77-ac58-931b4204f7be admin admin] PATH_INFO: `/v3/projects/3b75c9c598704bffa405dc9be4ffa9b9/users/2606be1252c44a87a990598b6d63d6b2/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:24.772988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06bb25-4859-4c77-ac58-931b4204f7be admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b75c9c598704bffa405dc9be4ffa9b9, user_id=2606be1252c44a87a990598b6d63d6b2, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:24.788107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06bb25-4859-4c77-ac58-931b4204f7be admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:24.937886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1044/2083] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:24 2022] PUT /identity/v3/projects/3b75c9c598704bffa405dc9be4ffa9b9/users/2606be1252c44a87a990598b6d63d6b2/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:24.959035 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ec209-c16f-4b31-b714-e32fe61e7bc7 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:24.971110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ec209-c16f-4b31-b714-e32fe61e7bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:24.971110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ec209-c16f-4b31-b714-e32fe61e7bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:25.019744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-5a5ec209-c16f-4b31-b714-e32fe61e7bc7 None None] MFA Rules not processed for user `2606be1252c44a87a990598b6d63d6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:25.022396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-5a5ec209-c16f-4b31-b714-e32fe61e7bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:25.139956 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1040/2084] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:25.152876 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06bb25-4859-4c77-ac58-931b4204f7be admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:25.255089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06bb25-4859-4c77-ac58-931b4204f7be admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:25.265203 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4447006-45ce-48b7-bc3b-975b9a16159a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:25.265896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4447006-45ce-48b7-bc3b-975b9a16159a admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:25.266180 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4447006-45ce-48b7-bc3b-975b9a16159a admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:25.266453 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4447006-45ce-48b7-bc3b-975b9a16159a admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:25.267219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4447006-45ce-48b7-bc3b-975b9a16159a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:25.269189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4447006-45ce-48b7-bc3b-975b9a16159a admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:25.825577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1045/2085] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:15:25 2022] POST /identity/v3/projects => generated 385 bytes in 677 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:25.827565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ec209-c16f-4b31-b714-e32fe61e7bc7 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:25.860097 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ec209-c16f-4b31-b714-e32fe61e7bc7 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:25.970816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9910d-7941-4e10-9be1-703143f7fc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:25.971860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9910d-7941-4e10-9be1-703143f7fc23 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:25.972525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9910d-7941-4e10-9be1-703143f7fc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:25.973005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9910d-7941-4e10-9be1-703143f7fc23 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:25.973916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9910d-7941-4e10-9be1-703143f7fc23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:25.977343 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9910d-7941-4e10-9be1-703143f7fc23 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:26.148928 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1041/2086] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:15:25 2022] POST /identity/v3/users => generated 299 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:15:26.153558 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4447006-45ce-48b7-bc3b-975b9a16159a admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:26.168617 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4447006-45ce-48b7-bc3b-975b9a16159a admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:26.174779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f000da-042f-4422-8444-fb4bb90ea811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:26.175840 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f000da-042f-4422-8444-fb4bb90ea811 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:26.176369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f000da-042f-4422-8444-fb4bb90ea811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:26.176846 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f000da-042f-4422-8444-fb4bb90ea811 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:26.177610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f000da-042f-4422-8444-fb4bb90ea811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:26.180588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f000da-042f-4422-8444-fb4bb90ea811 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:26.199530 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9910d-7941-4e10-9be1-703143f7fc23 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:26.200307 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1046/2087] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:26.215953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9910d-7941-4e10-9be1-703143f7fc23 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:26.223465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ee46f-ed61-44d6-9dce-7734cc92c9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:26.224870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ee46f-ed61-44d6-9dce-7734cc92c9c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:26.225492 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ee46f-ed61-44d6-9dce-7734cc92c9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:26.225950 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ee46f-ed61-44d6-9dce-7734cc92c9c0 admin admin] PATH_INFO: `/v3/projects/edb7dad9b7824254988eb995745aa879/users/b195ad7f5e074ec3af0da9d80f6854ff/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:26.252280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ee46f-ed61-44d6-9dce-7734cc92c9c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edb7dad9b7824254988eb995745aa879, user_id=b195ad7f5e074ec3af0da9d80f6854ff, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:26.255371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ee46f-ed61-44d6-9dce-7734cc92c9c0 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:26.377716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1042/2088] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:26 2022] PUT /identity/v3/projects/edb7dad9b7824254988eb995745aa879/users/b195ad7f5e074ec3af0da9d80f6854ff/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:26.379145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f000da-042f-4422-8444-fb4bb90ea811 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:26.394019 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f000da-042f-4422-8444-fb4bb90ea811 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:26.552936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82534f-1a2c-418e-b9b4-5d489fe26b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:26.552936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82534f-1a2c-418e-b9b4-5d489fe26b3e admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:26.552936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82534f-1a2c-418e-b9b4-5d489fe26b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:26.552936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82534f-1a2c-418e-b9b4-5d489fe26b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:26.554747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82534f-1a2c-418e-b9b4-5d489fe26b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:26.556706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82534f-1a2c-418e-b9b4-5d489fe26b3e admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:26.573067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1047/2089] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:15:26.580100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ee46f-ed61-44d6-9dce-7734cc92c9c0 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:26.601987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ee46f-ed61-44d6-9dce-7734cc92c9c0 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:26.613227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae9edb-50b7-4698-a995-d476190494e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:26.614260 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae9edb-50b7-4698-a995-d476190494e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:26.614701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae9edb-50b7-4698-a995-d476190494e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:26.615103 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae9edb-50b7-4698-a995-d476190494e4 admin admin] PATH_INFO: `/v3/projects/edb7dad9b7824254988eb995745aa879/users/b195ad7f5e074ec3af0da9d80f6854ff/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:26.619319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae9edb-50b7-4698-a995-d476190494e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edb7dad9b7824254988eb995745aa879, user_id=b195ad7f5e074ec3af0da9d80f6854ff, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:26.622347 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae9edb-50b7-4698-a995-d476190494e4 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:26.644447 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1043/2090] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:26 2022] PUT /identity/v3/projects/edb7dad9b7824254988eb995745aa879/users/b195ad7f5e074ec3af0da9d80f6854ff/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:15:26.666718 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81283f1f-b3a4-45c2-8ba4-2fa3507360d4 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:26.667371 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81283f1f-b3a4-45c2-8ba4-2fa3507360d4 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:26.667844 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81283f1f-b3a4-45c2-8ba4-2fa3507360d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:26.676715 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:26.762570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-81283f1f-b3a4-45c2-8ba4-2fa3507360d4 None None] MFA Rules not processed for user `b195ad7f5e074ec3af0da9d80f6854ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:26.763914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-81283f1f-b3a4-45c2-8ba4-2fa3507360d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:27.020403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1048/2091] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:27.036007 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae9edb-50b7-4698-a995-d476190494e4 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:27.044724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-5eae9edb-50b7-4698-a995-d476190494e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:27.094010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:27.226059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae9edb-50b7-4698-a995-d476190494e4 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:27.232295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62725c6-dfa8-4f48-b171-28d091846f9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:27.233075 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62725c6-dfa8-4f48-b171-28d091846f9d service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:27.233510 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62725c6-dfa8-4f48-b171-28d091846f9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:27.234265 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62725c6-dfa8-4f48-b171-28d091846f9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:27.254019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62725c6-dfa8-4f48-b171-28d091846f9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:27.257794 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62725c6-dfa8-4f48-b171-28d091846f9d service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:27.275393 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1044/2092] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:29.155642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61974241-fa89-406d-948d-694e9be71494 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:29.156221 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61974241-fa89-406d-948d-694e9be71494 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:29.156221 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61974241-fa89-406d-948d-694e9be71494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:29.184098 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-61974241-fa89-406d-948d-694e9be71494 None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:29.185364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-61974241-fa89-406d-948d-694e9be71494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:29.289720 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1049/2093] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Jul 18 15:15:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:29.305850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62725c6-dfa8-4f48-b171-28d091846f9d service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:29.309243 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c62725c6-dfa8-4f48-b171-28d091846f9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:29.395240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61974241-fa89-406d-948d-694e9be71494 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:29.427825 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61974241-fa89-406d-948d-694e9be71494 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:29.437325 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd3bfe-5b54-406f-9652-caf625bf90e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:29.438145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd3bfe-5b54-406f-9652-caf625bf90e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:29.438894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd3bfe-5b54-406f-9652-caf625bf90e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:29.439183 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd3bfe-5b54-406f-9652-caf625bf90e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:29.463593 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62725c6-dfa8-4f48-b171-28d091846f9d service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:29.499884 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5db966-ec30-4956-b09a-5f1b397742d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:29.500636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5db966-ec30-4956-b09a-5f1b397742d9 service swift] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:29.501091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5db966-ec30-4956-b09a-5f1b397742d9 service swift] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:29.501346 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5db966-ec30-4956-b09a-5f1b397742d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:29.519218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5db966-ec30-4956-b09a-5f1b397742d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:29.523328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5db966-ec30-4956-b09a-5f1b397742d9 service swift] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:29.552130 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd3bfe-5b54-406f-9652-caf625bf90e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:29.554937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd3bfe-5b54-406f-9652-caf625bf90e7 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:29.563102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1045/2094] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:15:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 15:15:29.591298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1050/2095] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:29.923280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5db966-ec30-4956-b09a-5f1b397742d9 service swift] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:29.923280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-4d5db966-ec30-4956-b09a-5f1b397742d9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:30.071771 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5db966-ec30-4956-b09a-5f1b397742d9 service swift] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:30.077278 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f0ad5-edaf-418d-a9c8-be08ffebdb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:30.078375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f0ad5-edaf-418d-a9c8-be08ffebdb94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:30.078709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f0ad5-edaf-418d-a9c8-be08ffebdb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:30.078998 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f0ad5-edaf-418d-a9c8-be08ffebdb94 admin admin] PATH_INFO: `/v3/users/6c85cf89ac304a9caeac72b5805d0dfc` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:30.080526 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f0ad5-edaf-418d-a9c8-be08ffebdb94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c85cf89ac304a9caeac72b5805d0dfc)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:30.082718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f0ad5-edaf-418d-a9c8-be08ffebdb94 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:30.709798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-6e9f0ad5-edaf-418d-a9c8-be08ffebdb94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c85cf89ac304a9caeac72b5805d0dfc'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:30.710830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-6e9f0ad5-edaf-418d-a9c8-be08ffebdb94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c85cf89ac304a9caeac72b5805d0dfc'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:30.711372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-6e9f0ad5-edaf-418d-a9c8-be08ffebdb94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c85cf89ac304a9caeac72b5805d0dfc'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:30.717384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:30.828279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-6e9f0ad5-edaf-418d-a9c8-be08ffebdb94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c85cf89ac304a9caeac72b5805d0dfc'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:30.849309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-6e9f0ad5-edaf-418d-a9c8-be08ffebdb94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c85cf89ac304a9caeac72b5805d0dfc'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:30.884107 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1046/2096] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:15:29 2022] DELETE /identity/v3/users/6c85cf89ac304a9caeac72b5805d0dfc => generated 0 bytes in 960 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:30.887295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f838e-1dec-4fa7-92c7-5d7d8d0b36b5 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:30.888263 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f838e-1dec-4fa7-92c7-5d7d8d0b36b5 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:30.888263 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f838e-1dec-4fa7-92c7-5d7d8d0b36b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:30.930233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-818f838e-1dec-4fa7-92c7-5d7d8d0b36b5 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:30.932339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-818f838e-1dec-4fa7-92c7-5d7d8d0b36b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:30.997827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:31.057411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1051/2097] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:31.080912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f0ad5-edaf-418d-a9c8-be08ffebdb94 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:31.082882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6e9f0ad5-edaf-418d-a9c8-be08ffebdb94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:31.280553 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f0ad5-edaf-418d-a9c8-be08ffebdb94 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:31.285104 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4210ea8-ff5f-405e-9211-d054397f246b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:31.286110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4210ea8-ff5f-405e-9211-d054397f246b service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:31.286412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4210ea8-ff5f-405e-9211-d054397f246b service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:31.286981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4210ea8-ff5f-405e-9211-d054397f246b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:31.315155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4210ea8-ff5f-405e-9211-d054397f246b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:31.320761 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4210ea8-ff5f-405e-9211-d054397f246b service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:31.345158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1047/2098] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:32.211629 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70570d31-3532-4df1-b338-0a211c52abfd None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:32.212764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70570d31-3532-4df1-b338-0a211c52abfd None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:32.213236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70570d31-3532-4df1-b338-0a211c52abfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:32.233305 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-70570d31-3532-4df1-b338-0a211c52abfd None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:32.234617 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-70570d31-3532-4df1-b338-0a211c52abfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:32.336847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1052/2099] 10.222.0.83 () {58 vars in 967 bytes} [Mon Jul 18 15:15:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:32.372319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4210ea8-ff5f-405e-9211-d054397f246b service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:32.375886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b4210ea8-ff5f-405e-9211-d054397f246b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:32.405985 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4210ea8-ff5f-405e-9211-d054397f246b service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:32.414629 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625580f2-cbb7-41f6-8dcf-79dadede95a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:32.416463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625580f2-cbb7-41f6-8dcf-79dadede95a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:32.417058 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625580f2-cbb7-41f6-8dcf-79dadede95a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:32.417767 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625580f2-cbb7-41f6-8dcf-79dadede95a1 admin admin] PATH_INFO: `/v3/projects/3087891c94f34fc2a6234f04c03f27e0` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:32.431327 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625580f2-cbb7-41f6-8dcf-79dadede95a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3087891c94f34fc2a6234f04c03f27e0)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:32.440079 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625580f2-cbb7-41f6-8dcf-79dadede95a1 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:32.447165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70570d31-3532-4df1-b338-0a211c52abfd None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:32.450849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-70570d31-3532-4df1-b338-0a211c52abfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:32.609215 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70570d31-3532-4df1-b338-0a211c52abfd None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:32.616252 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06279b88-369e-43fe-ac2b-c523e28c0bb0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:32.617203 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06279b88-369e-43fe-ac2b-c523e28c0bb0 service swift] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:32.617508 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06279b88-369e-43fe-ac2b-c523e28c0bb0 service swift] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:32.617788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06279b88-369e-43fe-ac2b-c523e28c0bb0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:32.635650 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06279b88-369e-43fe-ac2b-c523e28c0bb0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:32.635650 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06279b88-369e-43fe-ac2b-c523e28c0bb0 service swift] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:32.644374 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1053/2100] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:15:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 15:15:32.711549 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-625580f2-cbb7-41f6-8dcf-79dadede95a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3087891c94f34fc2a6234f04c03f27e0', 'request_id': 'req-625580f2-cbb7-41f6-8dcf-79dadede95a1'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:32.723236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-625580f2-cbb7-41f6-8dcf-79dadede95a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3087891c94f34fc2a6234f04c03f27e0', 'request_id': 'req-625580f2-cbb7-41f6-8dcf-79dadede95a1'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:32.732989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-625580f2-cbb7-41f6-8dcf-79dadede95a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3087891c94f34fc2a6234f04c03f27e0', 'request_id': 'req-625580f2-cbb7-41f6-8dcf-79dadede95a1'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:32.835214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-625580f2-cbb7-41f6-8dcf-79dadede95a1 admin admin] The token cache is being invalidate because project 3087891c94f34fc2a6234f04c03f27e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:15:32.835214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-625580f2-cbb7-41f6-8dcf-79dadede95a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:32.836074 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-625580f2-cbb7-41f6-8dcf-79dadede95a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:32.838482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1048/2101] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:15:32 2022] DELETE /identity/v3/projects/3087891c94f34fc2a6234f04c03f27e0 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:32.844079 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0570b-5df4-44bc-b7eb-500464514574 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:32.844427 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0570b-5df4-44bc-b7eb-500464514574 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:32.844644 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0570b-5df4-44bc-b7eb-500464514574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:32.884096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-82e0570b-5df4-44bc-b7eb-500464514574 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:32.887456 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-82e0570b-5df4-44bc-b7eb-500464514574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:33.101935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66593-beea-49be-b27b-5e6aede6a39a None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:33.102205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1054/2102] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:33.102830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66593-beea-49be-b27b-5e6aede6a39a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:33.103340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66593-beea-49be-b27b-5e6aede6a39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:33.126156 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-77e66593-beea-49be-b27b-5e6aede6a39a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:33.127698 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-77e66593-beea-49be-b27b-5e6aede6a39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:33.140604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1049/2103] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:33.144298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0570b-5df4-44bc-b7eb-500464514574 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:33.156677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0570b-5df4-44bc-b7eb-500464514574 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:33.166526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e64d7e-ed6a-4c34-8823-bc5b327f0180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:33.167260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e64d7e-ed6a-4c34-8823-bc5b327f0180 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:33.167553 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e64d7e-ed6a-4c34-8823-bc5b327f0180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:33.167840 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e64d7e-ed6a-4c34-8823-bc5b327f0180 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:33.168594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e64d7e-ed6a-4c34-8823-bc5b327f0180 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:15:33.168902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e64d7e-ed6a-4c34-8823-bc5b327f0180 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:33.170971 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e64d7e-ed6a-4c34-8823-bc5b327f0180 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:33.175129 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:33.192686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1055/2104] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 15:15:33.197314 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e66593-beea-49be-b27b-5e6aede6a39a None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:33.211286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e66593-beea-49be-b27b-5e6aede6a39a None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:33.217822 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f46006-1df8-42de-b508-aad24d471743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:33.219085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f46006-1df8-42de-b508-aad24d471743 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:33.219403 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f46006-1df8-42de-b508-aad24d471743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:33.219763 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f46006-1df8-42de-b508-aad24d471743 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:33.222600 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f46006-1df8-42de-b508-aad24d471743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:33.224567 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f46006-1df8-42de-b508-aad24d471743 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:33.438726 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1050/2105] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:15:33 2022] POST /identity/v3/projects => generated 369 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:33.443490 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e64d7e-ed6a-4c34-8823-bc5b327f0180 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:33.486834 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e64d7e-ed6a-4c34-8823-bc5b327f0180 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:33.661364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17257e-2c70-46ad-865a-3d4b7ef25cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:33.666253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17257e-2c70-46ad-865a-3d4b7ef25cd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:33.666253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17257e-2c70-46ad-865a-3d4b7ef25cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:33.667280 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17257e-2c70-46ad-865a-3d4b7ef25cd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:33.671108 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c17257e-2c70-46ad-865a-3d4b7ef25cd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:33.691652 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c17257e-2c70-46ad-865a-3d4b7ef25cd8 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:33.892186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f46006-1df8-42de-b508-aad24d471743 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:33.915237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1056/2106] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:15:33 2022] POST /identity/v3/users => generated 285 bytes in 473 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:33.939918 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f46006-1df8-42de-b508-aad24d471743 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:33.958011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b63e0-bf06-4b04-832c-d7c69eed1ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:33.959332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b63e0-bf06-4b04-832c-d7c69eed1ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:33.961336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b63e0-bf06-4b04-832c-d7c69eed1ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:33.961865 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b63e0-bf06-4b04-832c-d7c69eed1ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:33.963623 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b63e0-bf06-4b04-832c-d7c69eed1ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:33.970640 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b63e0-bf06-4b04-832c-d7c69eed1ae1 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:34.016704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17257e-2c70-46ad-865a-3d4b7ef25cd8 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:34.027810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1051/2107] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:34.041281 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17257e-2c70-46ad-865a-3d4b7ef25cd8 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:34.084221 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07103a-93f7-41cf-a984-62cfa033212c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:34.093179 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07103a-93f7-41cf-a984-62cfa033212c admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:34.094047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07103a-93f7-41cf-a984-62cfa033212c admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:34.094152 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07103a-93f7-41cf-a984-62cfa033212c admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:34.095566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07103a-93f7-41cf-a984-62cfa033212c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:34.102392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07103a-93f7-41cf-a984-62cfa033212c admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:34.131066 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b63e0-bf06-4b04-832c-d7c69eed1ae1 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:34.141105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1057/2108] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:15:34.152589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b63e0-bf06-4b04-832c-d7c69eed1ae1 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:34.161351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b9b19-e721-499b-8285-ee53453a5ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:34.164516 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b9b19-e721-499b-8285-ee53453a5ec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:34.164842 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b9b19-e721-499b-8285-ee53453a5ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:34.165206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b9b19-e721-499b-8285-ee53453a5ec8 admin admin] PATH_INFO: `/v3/projects/ee222427448b437d84d5ba9b19ec3afc/users/28b2d2ba51ea4e9f934403fc089fe96e/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:34.196301 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ecf62-903b-4141-a259-e23e65ad8db9 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:34.197326 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ecf62-903b-4141-a259-e23e65ad8db9 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:34.198082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ecf62-903b-4141-a259-e23e65ad8db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:34.255822 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:34.268074 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b9b19-e721-499b-8285-ee53453a5ec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee222427448b437d84d5ba9b19ec3afc, user_id=28b2d2ba51ea4e9f934403fc089fe96e, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:34.283580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b9b19-e721-499b-8285-ee53453a5ec8 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:34.286908 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-ba0ecf62-903b-4141-a259-e23e65ad8db9 None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:34.298203 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ba0ecf62-903b-4141-a259-e23e65ad8db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:34.462688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1052/2109] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:34 2022] PUT /identity/v3/projects/ee222427448b437d84d5ba9b19ec3afc/users/28b2d2ba51ea4e9f934403fc089fe96e/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:15:34.480745 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1058/2110] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Jul 18 15:15:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:34.486250 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17bc95-be14-44b7-87ab-a918dda0965f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:34.486250 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17bc95-be14-44b7-87ab-a918dda0965f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:34.486725 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17bc95-be14-44b7-87ab-a918dda0965f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:34.519335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ecf62-903b-4141-a259-e23e65ad8db9 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:34.521379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ba0ecf62-903b-4141-a259-e23e65ad8db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:34.639547 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-3a17bc95-be14-44b7-87ab-a918dda0965f None None] MFA Rules not processed for user `28b2d2ba51ea4e9f934403fc089fe96e`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:34.640754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-3a17bc95-be14-44b7-87ab-a918dda0965f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:34.813430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ecf62-903b-4141-a259-e23e65ad8db9 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:34.839114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:34.855727 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1053/2111] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:34.856973 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17bc95-be14-44b7-87ab-a918dda0965f None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:34.858344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-3a17bc95-be14-44b7-87ab-a918dda0965f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:34.911708 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0be040-47cd-4b0f-a9bd-7cb0562b0b2b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:34.913231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0be040-47cd-4b0f-a9bd-7cb0562b0b2b service swift] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:34.914519 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0be040-47cd-4b0f-a9bd-7cb0562b0b2b service swift] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:34.914937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0be040-47cd-4b0f-a9bd-7cb0562b0b2b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:34.966361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17bc95-be14-44b7-87ab-a918dda0965f None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:34.974440 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67536cec-ff25-4f9a-bde2-4b417b520a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:34.978524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67536cec-ff25-4f9a-bde2-4b417b520a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:34.986658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67536cec-ff25-4f9a-bde2-4b417b520a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:34.986658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67536cec-ff25-4f9a-bde2-4b417b520a05 admin admin] PATH_INFO: `/v3/users/9e7007a0ee8b4633a36efa32d10f46e4` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:34.988060 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67536cec-ff25-4f9a-bde2-4b417b520a05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e7007a0ee8b4633a36efa32d10f46e4)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:34.990199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67536cec-ff25-4f9a-bde2-4b417b520a05 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:35.019652 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0be040-47cd-4b0f-a9bd-7cb0562b0b2b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:35.021363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0be040-47cd-4b0f-a9bd-7cb0562b0b2b service swift] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:35.038379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1059/2112] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:15:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 15:15:35.039508 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0be040-47cd-4b0f-a9bd-7cb0562b0b2b service swift] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:35.040992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-5d0be040-47cd-4b0f-a9bd-7cb0562b0b2b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:35.158689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0be040-47cd-4b0f-a9bd-7cb0562b0b2b service swift] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:35.170843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c803738-787a-40cf-84e7-053740320e45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:35.171528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c803738-787a-40cf-84e7-053740320e45 service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:35.171832 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c803738-787a-40cf-84e7-053740320e45 service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:35.172127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c803738-787a-40cf-84e7-053740320e45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:35.181853 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c803738-787a-40cf-84e7-053740320e45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:35.183684 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c803738-787a-40cf-84e7-053740320e45 service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:35.195244 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1060/2113] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:35 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:35.772957 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-67536cec-ff25-4f9a-bde2-4b417b520a05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e7007a0ee8b4633a36efa32d10f46e4'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:35.774021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-67536cec-ff25-4f9a-bde2-4b417b520a05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e7007a0ee8b4633a36efa32d10f46e4'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:35.774763 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-67536cec-ff25-4f9a-bde2-4b417b520a05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e7007a0ee8b4633a36efa32d10f46e4'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:35.914820 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-67536cec-ff25-4f9a-bde2-4b417b520a05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e7007a0ee8b4633a36efa32d10f46e4'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:35.939334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-67536cec-ff25-4f9a-bde2-4b417b520a05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e7007a0ee8b4633a36efa32d10f46e4'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:35.949988 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:35.959700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1054/2114] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:15:34 2022] DELETE /identity/v3/users/9e7007a0ee8b4633a36efa32d10f46e4 => generated 0 bytes in 1103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:35.964808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f887681-5670-4f31-8626-fbc3bf532e72 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:35.965411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f887681-5670-4f31-8626-fbc3bf532e72 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:35.965411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f887681-5670-4f31-8626-fbc3bf532e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:35.991975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-8f887681-5670-4f31-8626-fbc3bf532e72 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:35.993492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-8f887681-5670-4f31-8626-fbc3bf532e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:36.071374 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:36.108055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1061/2115] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:36.121841 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67536cec-ff25-4f9a-bde2-4b417b520a05 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:36.123780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-67536cec-ff25-4f9a-bde2-4b417b520a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:36.216371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67536cec-ff25-4f9a-bde2-4b417b520a05 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:36.222085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022df4b-e1ee-4f31-b59d-ccc2be97c352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:36.222963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022df4b-e1ee-4f31-b59d-ccc2be97c352 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:36.223390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022df4b-e1ee-4f31-b59d-ccc2be97c352 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:36.223777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022df4b-e1ee-4f31-b59d-ccc2be97c352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:36.237796 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022df4b-e1ee-4f31-b59d-ccc2be97c352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:36.240310 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022df4b-e1ee-4f31-b59d-ccc2be97c352 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:36.257019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1055/2116] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:36.516757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0cc02-af78-43cf-80a7-a90846bef015 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:36.517196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0cc02-af78-43cf-80a7-a90846bef015 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:36.517551 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0cc02-af78-43cf-80a7-a90846bef015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:36.536890 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-b1e0cc02-af78-43cf-80a7-a90846bef015 None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:36.538188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b1e0cc02-af78-43cf-80a7-a90846bef015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:36.606455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1062/2117] 10.222.0.83 () {58 vars in 967 bytes} [Mon Jul 18 15:15:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:36.633346 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022df4b-e1ee-4f31-b59d-ccc2be97c352 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:36.633346 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-8022df4b-e1ee-4f31-b59d-ccc2be97c352 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:36.706791 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022df4b-e1ee-4f31-b59d-ccc2be97c352 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:36.714009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e23f4-e4c3-4adf-9b50-dc49086de764 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:36.714737 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e23f4-e4c3-4adf-9b50-dc49086de764 service swift] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:36.715052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e23f4-e4c3-4adf-9b50-dc49086de764 service swift] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:36.715330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e23f4-e4c3-4adf-9b50-dc49086de764 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:36.748883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e23f4-e4c3-4adf-9b50-dc49086de764 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:36.748883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e23f4-e4c3-4adf-9b50-dc49086de764 service swift] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:36.748883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1056/2118] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:15:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 15:15:37.107271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0cc02-af78-43cf-80a7-a90846bef015 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:37.107271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b1e0cc02-af78-43cf-80a7-a90846bef015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:37.128534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0cc02-af78-43cf-80a7-a90846bef015 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:37.137320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190e055-b8fc-4624-9770-2c19adf2bfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:37.138319 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190e055-b8fc-4624-9770-2c19adf2bfd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:37.138611 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190e055-b8fc-4624-9770-2c19adf2bfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:37.146791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190e055-b8fc-4624-9770-2c19adf2bfd4 admin admin] PATH_INFO: `/v3/projects/5134df78edfd4a13a7b781db30344d9d` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:37.146791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190e055-b8fc-4624-9770-2c19adf2bfd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5134df78edfd4a13a7b781db30344d9d)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:37.146791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190e055-b8fc-4624-9770-2c19adf2bfd4 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:37.318400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b190e055-b8fc-4624-9770-2c19adf2bfd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5134df78edfd4a13a7b781db30344d9d', 'request_id': 'req-b190e055-b8fc-4624-9770-2c19adf2bfd4'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:37.328315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b190e055-b8fc-4624-9770-2c19adf2bfd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5134df78edfd4a13a7b781db30344d9d', 'request_id': 'req-b190e055-b8fc-4624-9770-2c19adf2bfd4'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:37.417945 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33978f7e-3abf-45b3-9a00-e9650789a95c None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:37.419375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33978f7e-3abf-45b3-9a00-e9650789a95c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:37.419910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33978f7e-3abf-45b3-9a00-e9650789a95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:37.461503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-33978f7e-3abf-45b3-9a00-e9650789a95c None None] MFA Rules not processed for user `45b1df340f8a4102adf5f680d7e64139`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:37.464065 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-33978f7e-3abf-45b3-9a00-e9650789a95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:37.494132 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b190e055-b8fc-4624-9770-2c19adf2bfd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5134df78edfd4a13a7b781db30344d9d', 'request_id': 'req-b190e055-b8fc-4624-9770-2c19adf2bfd4'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:37.527753 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b190e055-b8fc-4624-9770-2c19adf2bfd4 admin admin] The token cache is being invalidate because project 5134df78edfd4a13a7b781db30344d9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:15:37.528066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b190e055-b8fc-4624-9770-2c19adf2bfd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:37.545673 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b190e055-b8fc-4624-9770-2c19adf2bfd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:37.545673 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1063/2119] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:15:37 2022] DELETE /identity/v3/projects/5134df78edfd4a13a7b781db30344d9d => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:37.551268 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78ca1-f6b7-425e-a633-35212b587499 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:37.551471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78ca1-f6b7-425e-a633-35212b587499 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:37.551652 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78ca1-f6b7-425e-a633-35212b587499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:37.599446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-08d78ca1-f6b7-425e-a633-35212b587499 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:37.602290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-08d78ca1-f6b7-425e-a633-35212b587499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:37.673112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1057/2120] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:15:37.716223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1064/2121] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:15:37.729480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630ed27-8555-4960-ac89-0e9d47ad733b None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:37.729655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630ed27-8555-4960-ac89-0e9d47ad733b None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:37.745057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630ed27-8555-4960-ac89-0e9d47ad733b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:37.757818 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-a630ed27-8555-4960-ac89-0e9d47ad733b None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:37.759462 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a630ed27-8555-4960-ac89-0e9d47ad733b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:37.770028 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1058/2122] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:37.778949 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d78ca1-f6b7-425e-a633-35212b587499 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:37.800792 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d78ca1-f6b7-425e-a633-35212b587499 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:37.810490 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34738a1b-2847-401e-8f71-2d901274ffd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:37.812195 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34738a1b-2847-401e-8f71-2d901274ffd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:37.812560 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34738a1b-2847-401e-8f71-2d901274ffd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:37.813137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34738a1b-2847-401e-8f71-2d901274ffd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:37.814742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34738a1b-2847-401e-8f71-2d901274ffd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:15:37.814927 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34738a1b-2847-401e-8f71-2d901274ffd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:37.817417 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34738a1b-2847-401e-8f71-2d901274ffd8 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:37.854596 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1065/2123] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:15:37.856966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630ed27-8555-4960-ac89-0e9d47ad733b None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:37.868525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630ed27-8555-4960-ac89-0e9d47ad733b None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:37.889383 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe2403-a83b-48e5-b35f-7295861e29f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:37.893270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe2403-a83b-48e5-b35f-7295861e29f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:37.894094 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe2403-a83b-48e5-b35f-7295861e29f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:37.894739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe2403-a83b-48e5-b35f-7295861e29f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:37.901611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe2403-a83b-48e5-b35f-7295861e29f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:37.905928 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe2403-a83b-48e5-b35f-7295861e29f5 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:38.073686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34738a1b-2847-401e-8f71-2d901274ffd8 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:38.073686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-34738a1b-2847-401e-8f71-2d901274ffd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:38.082174 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:38.160660 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1059/2124] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:15:37 2022] POST /identity/v3/projects => generated 373 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:38.166607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe2403-a83b-48e5-b35f-7295861e29f5 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:38.171435 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34738a1b-2847-401e-8f71-2d901274ffd8 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:38.181400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:38.237790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4eeb3-eefd-47fd-9b15-c97fad246a7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:38.238587 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4eeb3-eefd-47fd-9b15-c97fad246a7a service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:38.238841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4eeb3-eefd-47fd-9b15-c97fad246a7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:38.239120 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4eeb3-eefd-47fd-9b15-c97fad246a7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:38.241564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c5a4eeb3-eefd-47fd-9b15-c97fad246a7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:38.254333 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe2403-a83b-48e5-b35f-7295861e29f5 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:38.261366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e960e59-a901-4a98-9401-d89e51906efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:38.262748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e960e59-a901-4a98-9401-d89e51906efc admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:38.263245 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e960e59-a901-4a98-9401-d89e51906efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:38.263833 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e960e59-a901-4a98-9401-d89e51906efc admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:38.264557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e960e59-a901-4a98-9401-d89e51906efc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:38.266631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e960e59-a901-4a98-9401-d89e51906efc admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:38.312230 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4eeb3-eefd-47fd-9b15-c97fad246a7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:38.313832 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4eeb3-eefd-47fd-9b15-c97fad246a7a service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:38.325115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1066/2125] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:38.462881 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1060/2126] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:15:38 2022] POST /identity/v3/users => generated 287 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:15:38.466293 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4eeb3-eefd-47fd-9b15-c97fad246a7a service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:38.480739 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4eeb3-eefd-47fd-9b15-c97fad246a7a service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:38.485716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e5e50-9140-4bb3-8702-47a7ce1528aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:38.486517 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e5e50-9140-4bb3-8702-47a7ce1528aa admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:38.486834 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e5e50-9140-4bb3-8702-47a7ce1528aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:38.487123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e5e50-9140-4bb3-8702-47a7ce1528aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:38.488346 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e5e50-9140-4bb3-8702-47a7ce1528aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:38.489719 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e5e50-9140-4bb3-8702-47a7ce1528aa admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:38.501969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1067/2127] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:38.513334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e960e59-a901-4a98-9401-d89e51906efc admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:38.524175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e960e59-a901-4a98-9401-d89e51906efc admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:38.529087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858eff96-7dd6-4ce1-9884-e49a3c0f3d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:38.529771 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858eff96-7dd6-4ce1-9884-e49a3c0f3d5d admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:38.530075 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858eff96-7dd6-4ce1-9884-e49a3c0f3d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:38.530492 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858eff96-7dd6-4ce1-9884-e49a3c0f3d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:38.531349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858eff96-7dd6-4ce1-9884-e49a3c0f3d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:38.533073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858eff96-7dd6-4ce1-9884-e49a3c0f3d5d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:38.544777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1061/2128] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:38.551193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e5e50-9140-4bb3-8702-47a7ce1528aa admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:38.562062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e5e50-9140-4bb3-8702-47a7ce1528aa admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:38.567017 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20cdb03-dc97-4996-bdae-73637effc911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:38.567943 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cdb03-dc97-4996-bdae-73637effc911 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:38.568227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cdb03-dc97-4996-bdae-73637effc911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:38.568514 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cdb03-dc97-4996-bdae-73637effc911 admin admin] PATH_INFO: `/v3/projects/43a5c5beef1a42f0ac47a1f540e55c21/users/faa3eace09844298a4cf0373ddc74de1/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:38.588655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20cdb03-dc97-4996-bdae-73637effc911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43a5c5beef1a42f0ac47a1f540e55c21, user_id=faa3eace09844298a4cf0373ddc74de1, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:38.591710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20cdb03-dc97-4996-bdae-73637effc911 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:38.780185 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1068/2129] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:38 2022] PUT /identity/v3/projects/43a5c5beef1a42f0ac47a1f540e55c21/users/faa3eace09844298a4cf0373ddc74de1/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:38.785512 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abf31a-5c68-4626-a675-77f6112fa94f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:38.788120 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abf31a-5c68-4626-a675-77f6112fa94f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:38.788657 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abf31a-5c68-4626-a675-77f6112fa94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:38.848962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-07abf31a-5c68-4626-a675-77f6112fa94f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:38.853969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-07abf31a-5c68-4626-a675-77f6112fa94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:38.988553 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1062/2130] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:39.000706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20cdb03-dc97-4996-bdae-73637effc911 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:39.002599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b20cdb03-dc97-4996-bdae-73637effc911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:39.092895 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20cdb03-dc97-4996-bdae-73637effc911 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:39.099387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26f911-21ad-4036-ba50-63753c2700be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:39.101190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26f911-21ad-4036-ba50-63753c2700be service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:39.101709 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26f911-21ad-4036-ba50-63753c2700be service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:39.102208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26f911-21ad-4036-ba50-63753c2700be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:39.114984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26f911-21ad-4036-ba50-63753c2700be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:39.117155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26f911-21ad-4036-ba50-63753c2700be service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:39.128489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1069/2131] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:40.677353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abf31a-5c68-4626-a675-77f6112fa94f None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:40.679249 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-07abf31a-5c68-4626-a675-77f6112fa94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:40.692190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abf31a-5c68-4626-a675-77f6112fa94f None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:40.698308 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f3ff6-dc8c-4861-a322-85b1749dcd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:40.699062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f3ff6-dc8c-4861-a322-85b1749dcd49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:40.699520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f3ff6-dc8c-4861-a322-85b1749dcd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:40.699678 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f3ff6-dc8c-4861-a322-85b1749dcd49 admin admin] PATH_INFO: `/v3/users/28b2d2ba51ea4e9f934403fc089fe96e` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:40.704646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f3ff6-dc8c-4861-a322-85b1749dcd49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28b2d2ba51ea4e9f934403fc089fe96e)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:40.704646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f3ff6-dc8c-4861-a322-85b1749dcd49 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:40.936453 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:41.139321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-e29f3ff6-dc8c-4861-a322-85b1749dcd49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28b2d2ba51ea4e9f934403fc089fe96e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:41.140313 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-e29f3ff6-dc8c-4861-a322-85b1749dcd49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28b2d2ba51ea4e9f934403fc089fe96e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:41.141110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-e29f3ff6-dc8c-4861-a322-85b1749dcd49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28b2d2ba51ea4e9f934403fc089fe96e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:41.175045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26f911-21ad-4036-ba50-63753c2700be service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:41.176648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-cc26f911-21ad-4036-ba50-63753c2700be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:41.191531 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26f911-21ad-4036-ba50-63753c2700be service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:41.198061 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e8596-0335-4fbb-a8b1-e98330574ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:41.198993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e8596-0335-4fbb-a8b1-e98330574ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:41.199385 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e8596-0335-4fbb-a8b1-e98330574ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:41.199561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e8596-0335-4fbb-a8b1-e98330574ce1 admin admin] PATH_INFO: `/v3/users/db821aa375c2475181fd0c9d5807a51b` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:41.200871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e8596-0335-4fbb-a8b1-e98330574ce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db821aa375c2475181fd0c9d5807a51b)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:41.202966 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e8596-0335-4fbb-a8b1-e98330574ce1 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:41.286489 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-e29f3ff6-dc8c-4861-a322-85b1749dcd49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28b2d2ba51ea4e9f934403fc089fe96e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:41.307944 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-e29f3ff6-dc8c-4861-a322-85b1749dcd49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28b2d2ba51ea4e9f934403fc089fe96e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:41.339106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:41.345477 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1063/2132] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:15:40 2022] DELETE /identity/v3/users/28b2d2ba51ea4e9f934403fc089fe96e => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:41.347139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d5c59-8640-495a-9f10-817c4afba639 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:41.348067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d5c59-8640-495a-9f10-817c4afba639 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:41.348668 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d5c59-8640-495a-9f10-817c4afba639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:41.375937 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-7f4d5c59-8640-495a-9f10-817c4afba639 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:41.377288 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7f4d5c59-8640-495a-9f10-817c4afba639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:41.525461 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-666e8596-0335-4fbb-a8b1-e98330574ce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db821aa375c2475181fd0c9d5807a51b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:41.533389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-666e8596-0335-4fbb-a8b1-e98330574ce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db821aa375c2475181fd0c9d5807a51b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:41.534640 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-666e8596-0335-4fbb-a8b1-e98330574ce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db821aa375c2475181fd0c9d5807a51b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:41.549875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-666e8596-0335-4fbb-a8b1-e98330574ce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db821aa375c2475181fd0c9d5807a51b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:41.552185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1064/2133] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:41.563593 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d5c59-8640-495a-9f10-817c4afba639 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:41.565344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7f4d5c59-8640-495a-9f10-817c4afba639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:41.668422 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d5c59-8640-495a-9f10-817c4afba639 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:41.668726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-666e8596-0335-4fbb-a8b1-e98330574ce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db821aa375c2475181fd0c9d5807a51b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:41.673653 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f24e7-e1dd-4080-87af-e731e161d44e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:41.674322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f24e7-e1dd-4080-87af-e731e161d44e service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:41.674946 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f24e7-e1dd-4080-87af-e731e161d44e service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:41.675258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f24e7-e1dd-4080-87af-e731e161d44e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:41.676354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1070/2134] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:15:41 2022] DELETE /identity/v3/users/db821aa375c2475181fd0c9d5807a51b => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:15:41.678712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7b03a-702d-49ee-b314-0966e2a43c6f None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:41.678712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7b03a-702d-49ee-b314-0966e2a43c6f None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:41.678712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7b03a-702d-49ee-b314-0966e2a43c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:41.679426 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-8e9f24e7-e1dd-4080-87af-e731e161d44e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:41.693263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f24e7-e1dd-4080-87af-e731e161d44e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:41.696647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f24e7-e1dd-4080-87af-e731e161d44e service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:41.710009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-28e7b03a-702d-49ee-b314-0966e2a43c6f None None] MFA Rules not processed for user `faa3eace09844298a4cf0373ddc74de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:41.711098 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1065/2135] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:41.711567 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-28e7b03a-702d-49ee-b314-0966e2a43c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:41.712768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e170473-2f22-4691-bb5b-c05dfb897ac7 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:41.713209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e170473-2f22-4691-bb5b-c05dfb897ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:41.713354 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e170473-2f22-4691-bb5b-c05dfb897ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:41.732990 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-1e170473-2f22-4691-bb5b-c05dfb897ac7 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:41.734261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-1e170473-2f22-4691-bb5b-c05dfb897ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:41.805831 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1071/2136] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:41.816936 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1066/2137] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:41.819524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7b03a-702d-49ee-b314-0966e2a43c6f None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:41.821418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-28e7b03a-702d-49ee-b314-0966e2a43c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:41.829923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e170473-2f22-4691-bb5b-c05dfb897ac7 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:41.831717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-1e170473-2f22-4691-bb5b-c05dfb897ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:41.901663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7b03a-702d-49ee-b314-0966e2a43c6f None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:41.905528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51e246-b47b-4efc-b2d6-1722f7cce0a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:41.906192 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51e246-b47b-4efc-b2d6-1722f7cce0a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:41.906483 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51e246-b47b-4efc-b2d6-1722f7cce0a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:41.906931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51e246-b47b-4efc-b2d6-1722f7cce0a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:41.909984 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e170473-2f22-4691-bb5b-c05dfb897ac7 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:41.914588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47954fd2-c3a1-435b-a711-a7d47a6ff86f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:41.915259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47954fd2-c3a1-435b-a711-a7d47a6ff86f service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:41.915542 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47954fd2-c3a1-435b-a711-a7d47a6ff86f service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:41.915987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47954fd2-c3a1-435b-a711-a7d47a6ff86f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:41.917540 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51e246-b47b-4efc-b2d6-1722f7cce0a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:41.919255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51e246-b47b-4efc-b2d6-1722f7cce0a5 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:41.927765 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47954fd2-c3a1-435b-a711-a7d47a6ff86f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:41.930311 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1072/2138] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:15:41.930454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47954fd2-c3a1-435b-a711-a7d47a6ff86f service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:41.941350 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1067/2139] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:15:42.344390 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51e246-b47b-4efc-b2d6-1722f7cce0a5 service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:42.345932 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ca51e246-b47b-4efc-b2d6-1722f7cce0a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:42.364312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51e246-b47b-4efc-b2d6-1722f7cce0a5 service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:42.370399 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f3dd6-8d88-4d7f-acf7-59d2921104ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:42.371414 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f3dd6-8d88-4d7f-acf7-59d2921104ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:42.371773 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f3dd6-8d88-4d7f-acf7-59d2921104ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:42.372205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f3dd6-8d88-4d7f-acf7-59d2921104ac admin admin] PATH_INFO: `/v3/projects/afc824f35b8144398ace513ad018ec98` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:42.373763 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f3dd6-8d88-4d7f-acf7-59d2921104ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=afc824f35b8144398ace513ad018ec98)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:42.375766 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f3dd6-8d88-4d7f-acf7-59d2921104ac admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:42.473503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47954fd2-c3a1-435b-a711-a7d47a6ff86f service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:42.483533 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-47954fd2-c3a1-435b-a711-a7d47a6ff86f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:42.528044 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47954fd2-c3a1-435b-a711-a7d47a6ff86f service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:42.542064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c40f3dd6-8d88-4d7f-acf7-59d2921104ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afc824f35b8144398ace513ad018ec98', 'request_id': 'req-c40f3dd6-8d88-4d7f-acf7-59d2921104ac'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:42.551491 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c40f3dd6-8d88-4d7f-acf7-59d2921104ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afc824f35b8144398ace513ad018ec98', 'request_id': 'req-c40f3dd6-8d88-4d7f-acf7-59d2921104ac'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:42.615000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147a73c-7122-4438-accb-e8031bd96477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:42.617029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147a73c-7122-4438-accb-e8031bd96477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:42.617029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147a73c-7122-4438-accb-e8031bd96477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:42.617219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147a73c-7122-4438-accb-e8031bd96477 admin admin] PATH_INFO: `/v3/projects/ee222427448b437d84d5ba9b19ec3afc` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:42.618485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147a73c-7122-4438-accb-e8031bd96477 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee222427448b437d84d5ba9b19ec3afc)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:42.620524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147a73c-7122-4438-accb-e8031bd96477 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:42.727927 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c40f3dd6-8d88-4d7f-acf7-59d2921104ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afc824f35b8144398ace513ad018ec98', 'request_id': 'req-c40f3dd6-8d88-4d7f-acf7-59d2921104ac'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:42.748909 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c40f3dd6-8d88-4d7f-acf7-59d2921104ac admin admin] The token cache is being invalidate because project afc824f35b8144398ace513ad018ec98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:15:42.749422 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c40f3dd6-8d88-4d7f-acf7-59d2921104ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:42.759288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c40f3dd6-8d88-4d7f-acf7-59d2921104ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:42.765055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1073/2140] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:15:42 2022] DELETE /identity/v3/projects/afc824f35b8144398ace513ad018ec98 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:42.766966 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ccc1e-3338-4d33-a008-4a7d3d0a4e66 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:42.767247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ccc1e-3338-4d33-a008-4a7d3d0a4e66 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:42.767706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ccc1e-3338-4d33-a008-4a7d3d0a4e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:42.789455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-482ccc1e-3338-4d33-a008-4a7d3d0a4e66 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:42.790985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-482ccc1e-3338-4d33-a008-4a7d3d0a4e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:42.795432 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:42.805076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1074/2141] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:42.814801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ff430-2d87-4cd6-8747-a26e2059e1d4 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:42.815393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ff430-2d87-4cd6-8747-a26e2059e1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:42.815777 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ff430-2d87-4cd6-8747-a26e2059e1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:42.827234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-f147a73c-7122-4438-accb-e8031bd96477 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee222427448b437d84d5ba9b19ec3afc', 'request_id': 'req-f147a73c-7122-4438-accb-e8031bd96477'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:42.833625 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-f147a73c-7122-4438-accb-e8031bd96477 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee222427448b437d84d5ba9b19ec3afc', 'request_id': 'req-f147a73c-7122-4438-accb-e8031bd96477'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:42.841725 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-f147a73c-7122-4438-accb-e8031bd96477 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee222427448b437d84d5ba9b19ec3afc', 'request_id': 'req-f147a73c-7122-4438-accb-e8031bd96477'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:42.849019 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-472ff430-2d87-4cd6-8747-a26e2059e1d4 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:42.852753 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-472ff430-2d87-4cd6-8747-a26e2059e1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:42.917256 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1075/2142] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:42.921620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ff430-2d87-4cd6-8747-a26e2059e1d4 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:42.931861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ff430-2d87-4cd6-8747-a26e2059e1d4 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:42.937441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-f147a73c-7122-4438-accb-e8031bd96477 admin admin] The token cache is being invalidate because project ee222427448b437d84d5ba9b19ec3afc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:15:42.937630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcbfac-8a9d-4f29-8db5-af872226d4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:42.938146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-f147a73c-7122-4438-accb-e8031bd96477 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:42.938282 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcbfac-8a9d-4f29-8db5-af872226d4bd admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:42.938582 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcbfac-8a9d-4f29-8db5-af872226d4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:42.938866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcbfac-8a9d-4f29-8db5-af872226d4bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:42.939014 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-f147a73c-7122-4438-accb-e8031bd96477 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:42.939599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dcbfac-8a9d-4f29-8db5-af872226d4bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:15:42.939908 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dcbfac-8a9d-4f29-8db5-af872226d4bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:42.940962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1068/2143] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:15:42 2022] DELETE /identity/v3/projects/ee222427448b437d84d5ba9b19ec3afc => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:42.941556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dcbfac-8a9d-4f29-8db5-af872226d4bd admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:42.948678 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020610c-b81a-4f42-b820-ab24f24d153d None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:42.949221 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020610c-b81a-4f42-b820-ab24f24d153d None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:42.950247 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020610c-b81a-4f42-b820-ab24f24d153d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:42.958986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1076/2144] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 15:15:42.960239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcbfac-8a9d-4f29-8db5-af872226d4bd admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:42.963565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f6dcbfac-8a9d-4f29-8db5-af872226d4bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:42.978760 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcbfac-8a9d-4f29-8db5-af872226d4bd admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:42.984584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421424ab-f8a8-4355-bf7b-1a9aaee35e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:42.984813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421424ab-f8a8-4355-bf7b-1a9aaee35e0c admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:42.985045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421424ab-f8a8-4355-bf7b-1a9aaee35e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:42.985326 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421424ab-f8a8-4355-bf7b-1a9aaee35e0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:42.986187 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421424ab-f8a8-4355-bf7b-1a9aaee35e0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:42.988271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421424ab-f8a8-4355-bf7b-1a9aaee35e0c admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:42.990514 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-7020610c-b81a-4f42-b820-ab24f24d153d None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:42.992941 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7020610c-b81a-4f42-b820-ab24f24d153d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:43.011100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1069/2145] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:15:43.015078 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a86357-90d3-46eb-8ee9-aabccd65dd04 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:43.015813 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a86357-90d3-46eb-8ee9-aabccd65dd04 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:43.016178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a86357-90d3-46eb-8ee9-aabccd65dd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:43.041052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b1a86357-90d3-46eb-8ee9-aabccd65dd04 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:43.042287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b1a86357-90d3-46eb-8ee9-aabccd65dd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:43.066450 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1070/2146] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:43.070759 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a86357-90d3-46eb-8ee9-aabccd65dd04 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:43.080540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a86357-90d3-46eb-8ee9-aabccd65dd04 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:43.091060 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c59a0-93f7-41a0-952d-dcd14851cbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:43.091648 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c59a0-93f7-41a0-952d-dcd14851cbad admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:43.093112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c59a0-93f7-41a0-952d-dcd14851cbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:43.093112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c59a0-93f7-41a0-952d-dcd14851cbad admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:43.093112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c59a0-93f7-41a0-952d-dcd14851cbad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:15:43.093655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c59a0-93f7-41a0-952d-dcd14851cbad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:43.096171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c59a0-93f7-41a0-952d-dcd14851cbad admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:43.114314 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1071/2147] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 15:15:43.118880 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c59a0-93f7-41a0-952d-dcd14851cbad admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:43.130082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c59a0-93f7-41a0-952d-dcd14851cbad admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:43.135288 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce13e3-1cac-44b3-a90a-329939a983f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:43.135920 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce13e3-1cac-44b3-a90a-329939a983f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:43.136211 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce13e3-1cac-44b3-a90a-329939a983f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:43.136493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce13e3-1cac-44b3-a90a-329939a983f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:43.137208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce13e3-1cac-44b3-a90a-329939a983f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:43.139232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce13e3-1cac-44b3-a90a-329939a983f8 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:43.156525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1077/2148] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:15:42 2022] POST /identity/v3/projects => generated 373 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:43.158150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421424ab-f8a8-4355-bf7b-1a9aaee35e0c admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:43.270064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421424ab-f8a8-4355-bf7b-1a9aaee35e0c admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:43.279030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94cdb0-3972-405b-a1fd-400e0a707f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:43.280579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94cdb0-3972-405b-a1fd-400e0a707f6f admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:43.281140 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94cdb0-3972-405b-a1fd-400e0a707f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:43.281741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94cdb0-3972-405b-a1fd-400e0a707f6f admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:43.283152 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad94cdb0-3972-405b-a1fd-400e0a707f6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:43.286878 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad94cdb0-3972-405b-a1fd-400e0a707f6f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:43.312109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1072/2149] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:15:43 2022] POST /identity/v3/projects => generated 405 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:15:43.318599 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce13e3-1cac-44b3-a90a-329939a983f8 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:43.406187 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce13e3-1cac-44b3-a90a-329939a983f8 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:43.414429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b33ba-d4d4-4e90-9699-afcae6ba9b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:43.415442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b33ba-d4d4-4e90-9699-afcae6ba9b2e admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:43.419281 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b33ba-d4d4-4e90-9699-afcae6ba9b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:43.419281 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b33ba-d4d4-4e90-9699-afcae6ba9b2e admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:43.419281 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b33ba-d4d4-4e90-9699-afcae6ba9b2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:43.419281 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b33ba-d4d4-4e90-9699-afcae6ba9b2e admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:43.452412 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1078/2150] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:15:43 2022] POST /identity/v3/users => generated 287 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:43.457572 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94cdb0-3972-405b-a1fd-400e0a707f6f admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:43.471168 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94cdb0-3972-405b-a1fd-400e0a707f6f admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:43.478124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242796d-9683-47df-8634-e73803a5dfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:43.478858 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242796d-9683-47df-8634-e73803a5dfa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:43.479144 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242796d-9683-47df-8634-e73803a5dfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:43.479456 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242796d-9683-47df-8634-e73803a5dfa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:43.480402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242796d-9683-47df-8634-e73803a5dfa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:43.482045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242796d-9683-47df-8634-e73803a5dfa9 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:43.495077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1079/2151] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:43.499378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242796d-9683-47df-8634-e73803a5dfa9 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:43.513361 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242796d-9683-47df-8634-e73803a5dfa9 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:43.523600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd059452-8a14-43fb-9e48-f317aa7f5617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:43.524620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd059452-8a14-43fb-9e48-f317aa7f5617 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:43.524826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd059452-8a14-43fb-9e48-f317aa7f5617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:43.525192 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd059452-8a14-43fb-9e48-f317aa7f5617 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:43.526030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd059452-8a14-43fb-9e48-f317aa7f5617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:43.528258 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd059452-8a14-43fb-9e48-f317aa7f5617 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:43.542002 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1080/2152] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:43.552451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd059452-8a14-43fb-9e48-f317aa7f5617 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:43.568441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd059452-8a14-43fb-9e48-f317aa7f5617 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:43.574468 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2236348-5036-4f1b-8b65-78571e08504f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:43.575567 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2236348-5036-4f1b-8b65-78571e08504f admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:43.575935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2236348-5036-4f1b-8b65-78571e08504f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:43.576334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2236348-5036-4f1b-8b65-78571e08504f admin admin] PATH_INFO: `/v3/projects/5c88f3beb8bf468f9a30311b203b13cf/users/dcfdd71f9b93426bb6b149d2aeb2088e/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:43.597571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1073/2153] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:15:43 2022] POST /identity/v3/users => generated 303 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:43.598831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b33ba-d4d4-4e90-9699-afcae6ba9b2e admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:43.610024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2236348-5036-4f1b-8b65-78571e08504f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c88f3beb8bf468f9a30311b203b13cf, user_id=dcfdd71f9b93426bb6b149d2aeb2088e, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:43.613839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2236348-5036-4f1b-8b65-78571e08504f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:43.628800 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b33ba-d4d4-4e90-9699-afcae6ba9b2e admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:43.637410 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b887b-5aa0-4ff4-ac49-8d3ffe793490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:43.639139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b887b-5aa0-4ff4-ac49-8d3ffe793490 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:43.640807 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b887b-5aa0-4ff4-ac49-8d3ffe793490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:43.641354 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b887b-5aa0-4ff4-ac49-8d3ffe793490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:43.642610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b887b-5aa0-4ff4-ac49-8d3ffe793490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:43.645838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b887b-5aa0-4ff4-ac49-8d3ffe793490 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:43.671437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1074/2154] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:43.672931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b887b-5aa0-4ff4-ac49-8d3ffe793490 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:43.689041 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b887b-5aa0-4ff4-ac49-8d3ffe793490 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:43.693004 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce46c7b-1b9f-40ed-9e3d-346868242f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:43.693952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce46c7b-1b9f-40ed-9e3d-346868242f29 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:43.694321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce46c7b-1b9f-40ed-9e3d-346868242f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:43.694651 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce46c7b-1b9f-40ed-9e3d-346868242f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:43.695484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce46c7b-1b9f-40ed-9e3d-346868242f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:43.697540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce46c7b-1b9f-40ed-9e3d-346868242f29 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:43.715079 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1075/2155] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:43.717402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce46c7b-1b9f-40ed-9e3d-346868242f29 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:43.742523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce46c7b-1b9f-40ed-9e3d-346868242f29 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:43.749837 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1081/2156] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:43 2022] PUT /identity/v3/projects/5c88f3beb8bf468f9a30311b203b13cf/users/dcfdd71f9b93426bb6b149d2aeb2088e/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:15:43.758211 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a6dbb-c91e-4c0a-9871-71d8feb1a8f1 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:43.758511 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a6dbb-c91e-4c0a-9871-71d8feb1a8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:43.759103 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a6dbb-c91e-4c0a-9871-71d8feb1a8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:43.806593 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-113a6dbb-c91e-4c0a-9871-71d8feb1a8f1 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:43.808061 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-113a6dbb-c91e-4c0a-9871-71d8feb1a8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:43.875304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a85442-9a23-451f-ae85-c5d7520ea14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:43.878097 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a85442-9a23-451f-ae85-c5d7520ea14c admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:43.881055 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a85442-9a23-451f-ae85-c5d7520ea14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:43.882599 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a85442-9a23-451f-ae85-c5d7520ea14c admin admin] PATH_INFO: `/v3/projects/305c3698fba14a46b027e557fb68d7d7/users/09d64d2cc7e64e0497e3c571aa8d9c89/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:43.907560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a85442-9a23-451f-ae85-c5d7520ea14c admin admin] RBAC: Authorizing `identity:create_grant(project_id=305c3698fba14a46b027e557fb68d7d7, user_id=09d64d2cc7e64e0497e3c571aa8d9c89, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:43.911956 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a85442-9a23-451f-ae85-c5d7520ea14c admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:43.938626 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1082/2157] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:15:43.948155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a68b98-3b32-48ee-bf9a-cea4639c3ce6 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:43.948729 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a68b98-3b32-48ee-bf9a-cea4639c3ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:43.949242 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a68b98-3b32-48ee-bf9a-cea4639c3ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:43.970784 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-f3a68b98-3b32-48ee-bf9a-cea4639c3ce6 None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:43.972057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f3a68b98-3b32-48ee-bf9a-cea4639c3ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:44.012011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1076/2158] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:43 2022] PUT /identity/v3/projects/305c3698fba14a46b027e557fb68d7d7/users/09d64d2cc7e64e0497e3c571aa8d9c89/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:44.012824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a85442-9a23-451f-ae85-c5d7520ea14c admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:44.015310 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-97a85442-9a23-451f-ae85-c5d7520ea14c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:44.110790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1083/2159] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Jul 18 15:15:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:44.112559 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daad52-d093-420e-b950-6eee4ffd21e6 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:44.113216 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daad52-d093-420e-b950-6eee4ffd21e6 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:44.113881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daad52-d093-420e-b950-6eee4ffd21e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:44.120636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a85442-9a23-451f-ae85-c5d7520ea14c admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:44.125197 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fda57-3f62-4207-8baa-a27fddbe442a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:44.126090 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fda57-3f62-4207-8baa-a27fddbe442a service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:44.126552 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fda57-3f62-4207-8baa-a27fddbe442a service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:44.128377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fda57-3f62-4207-8baa-a27fddbe442a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:44.158614 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:44.216298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-e9daad52-d093-420e-b950-6eee4ffd21e6 None None] MFA Rules not processed for user `09d64d2cc7e64e0497e3c571aa8d9c89`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:44.220881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e9daad52-d093-420e-b950-6eee4ffd21e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:44.331258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960fda57-3f62-4207-8baa-a27fddbe442a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:44.337042 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960fda57-3f62-4207-8baa-a27fddbe442a service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:44.357394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1084/2160] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:44 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:44.358759 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9daad52-d093-420e-b950-6eee4ffd21e6 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:44.359952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e9daad52-d093-420e-b950-6eee4ffd21e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:44.373070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1077/2161] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:44.374852 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beec0c9-b1af-4d5b-abbc-8c2aaccf508d None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:44.375168 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beec0c9-b1af-4d5b-abbc-8c2aaccf508d None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:44.375464 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beec0c9-b1af-4d5b-abbc-8c2aaccf508d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:44.435290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-1beec0c9-b1af-4d5b-abbc-8c2aaccf508d None None] MFA Rules not processed for user `faa3eace09844298a4cf0373ddc74de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:44.436595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-1beec0c9-b1af-4d5b-abbc-8c2aaccf508d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:44.468001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9daad52-d093-420e-b950-6eee4ffd21e6 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:44.474247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cdb8c-6d94-43d3-99c9-801241805e61 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:44.475960 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cdb8c-6d94-43d3-99c9-801241805e61 service swift] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:44.476422 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cdb8c-6d94-43d3-99c9-801241805e61 service swift] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:44.476861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cdb8c-6d94-43d3-99c9-801241805e61 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:44.538987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1078/2162] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:44.540072 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beec0c9-b1af-4d5b-abbc-8c2aaccf508d None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:44.552660 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beec0c9-b1af-4d5b-abbc-8c2aaccf508d None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:44.561190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c4fca-d98d-4fbc-86a0-f9da24e299a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:44.562132 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c4fca-d98d-4fbc-86a0-f9da24e299a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:44.562132 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c4fca-d98d-4fbc-86a0-f9da24e299a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:44.562629 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c4fca-d98d-4fbc-86a0-f9da24e299a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:44.563922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c4fca-d98d-4fbc-86a0-f9da24e299a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:44.567960 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c4fca-d98d-4fbc-86a0-f9da24e299a9 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:44.618387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54cdb8c-6d94-43d3-99c9-801241805e61 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:44.620341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54cdb8c-6d94-43d3-99c9-801241805e61 service swift] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:44.630410 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1085/2163] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:15:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 15:15:44.631982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cdb8c-6d94-43d3-99c9-801241805e61 service swift] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:44.635098 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a54cdb8c-6d94-43d3-99c9-801241805e61 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:44.650946 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cdb8c-6d94-43d3-99c9-801241805e61 service swift] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:44.659103 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85f7a1-18a6-4529-a1d4-f4298081841c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:44.663314 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85f7a1-18a6-4529-a1d4-f4298081841c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:44.664023 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85f7a1-18a6-4529-a1d4-f4298081841c admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:44.664446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85f7a1-18a6-4529-a1d4-f4298081841c admin admin] PATH_INFO: `/v3/users/331e27de24d248a8b97e2f008d890e02` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:44.666295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85f7a1-18a6-4529-a1d4-f4298081841c admin admin] RBAC: Authorizing `identity:delete_user(user_id=331e27de24d248a8b97e2f008d890e02)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:44.668151 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85f7a1-18a6-4529-a1d4-f4298081841c admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:44.758596 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1079/2164] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:15:44 2022] POST /identity/v3/projects => generated 403 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:44.764998 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c4fca-d98d-4fbc-86a0-f9da24e299a9 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:44.857638 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c4fca-d98d-4fbc-86a0-f9da24e299a9 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:44.867300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f81416-2903-4b25-a1a3-2861f649ca6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:44.868392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f81416-2903-4b25-a1a3-2861f649ca6f admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:44.871637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f81416-2903-4b25-a1a3-2861f649ca6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:44.871637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f81416-2903-4b25-a1a3-2861f649ca6f admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:44.871637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f81416-2903-4b25-a1a3-2861f649ca6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:44.872848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f81416-2903-4b25-a1a3-2861f649ca6f admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:45.031489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1e85f7a1-18a6-4529-a1d4-f4298081841c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '331e27de24d248a8b97e2f008d890e02'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:45.034083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1080/2165] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:15:44 2022] POST /identity/v3/users => generated 308 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:15:45.039693 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f81416-2903-4b25-a1a3-2861f649ca6f admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:45.083694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1e85f7a1-18a6-4529-a1d4-f4298081841c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '331e27de24d248a8b97e2f008d890e02'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:45.084695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1e85f7a1-18a6-4529-a1d4-f4298081841c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '331e27de24d248a8b97e2f008d890e02'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:45.090081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f81416-2903-4b25-a1a3-2861f649ca6f admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:45.103594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:45.114090 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc351163-ce78-4dd6-8930-a06f484ae9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:45.114829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc351163-ce78-4dd6-8930-a06f484ae9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:45.115125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc351163-ce78-4dd6-8930-a06f484ae9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:45.115414 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc351163-ce78-4dd6-8930-a06f484ae9e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:45.116735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc351163-ce78-4dd6-8930-a06f484ae9e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:45.120031 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc351163-ce78-4dd6-8930-a06f484ae9e9 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:45.127698 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1e85f7a1-18a6-4529-a1d4-f4298081841c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '331e27de24d248a8b97e2f008d890e02'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:45.145913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1081/2166] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:45.147012 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc351163-ce78-4dd6-8930-a06f484ae9e9 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:45.160766 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc351163-ce78-4dd6-8930-a06f484ae9e9 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:45.171509 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ea09b-789f-4dec-a74f-fe1c23c86c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:45.172440 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ea09b-789f-4dec-a74f-fe1c23c86c9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:45.173757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ea09b-789f-4dec-a74f-fe1c23c86c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:45.175178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ea09b-789f-4dec-a74f-fe1c23c86c9c admin admin] PATH_INFO: `/v3/projects/8602e61d940c4578b8377741775dfe62/users/ef8a5e8073e942a29b6e33933b61ecf3/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:45.209465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ea09b-789f-4dec-a74f-fe1c23c86c9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8602e61d940c4578b8377741775dfe62, user_id=ef8a5e8073e942a29b6e33933b61ecf3, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:45.214814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ea09b-789f-4dec-a74f-fe1c23c86c9c admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:45.223829 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1e85f7a1-18a6-4529-a1d4-f4298081841c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '331e27de24d248a8b97e2f008d890e02'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:45.247272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1086/2167] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:15:44 2022] DELETE /identity/v3/users/331e27de24d248a8b97e2f008d890e02 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:45.251642 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad2c9b-ec3d-49c5-9cb0-397c6a54aa57 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:45.251935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad2c9b-ec3d-49c5-9cb0-397c6a54aa57 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:45.252300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad2c9b-ec3d-49c5-9cb0-397c6a54aa57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:45.309794 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-10ad2c9b-ec3d-49c5-9cb0-397c6a54aa57 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:45.311488 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-10ad2c9b-ec3d-49c5-9cb0-397c6a54aa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:45.318018 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1082/2168] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:45 2022] PUT /identity/v3/projects/8602e61d940c4578b8377741775dfe62/users/ef8a5e8073e942a29b6e33933b61ecf3/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:45.323531 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ea09b-789f-4dec-a74f-fe1c23c86c9c admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:45.343300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6f7ea09b-789f-4dec-a74f-fe1c23c86c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:45.452466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:45.541522 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ea09b-789f-4dec-a74f-fe1c23c86c9c admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:45.549533 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6d385-90d4-4a3c-813b-55e5dc3f607c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:45.550790 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6d385-90d4-4a3c-813b-55e5dc3f607c admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:45.551265 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6d385-90d4-4a3c-813b-55e5dc3f607c admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:45.552599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1087/2169] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:15:45.553228 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6d385-90d4-4a3c-813b-55e5dc3f607c admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:45.558366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6d385-90d4-4a3c-813b-55e5dc3f607c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:45.560297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6d385-90d4-4a3c-813b-55e5dc3f607c admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:45.567654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad2c9b-ec3d-49c5-9cb0-397c6a54aa57 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:45.575836 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-10ad2c9b-ec3d-49c5-9cb0-397c6a54aa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:45.598063 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1083/2170] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:45.602764 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6d385-90d4-4a3c-813b-55e5dc3f607c admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:45.658510 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6d385-90d4-4a3c-813b-55e5dc3f607c admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:45.694677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df7b5b-b6d0-4306-adfc-e1a17eb61bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:45.695794 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df7b5b-b6d0-4306-adfc-e1a17eb61bf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:45.696087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df7b5b-b6d0-4306-adfc-e1a17eb61bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:45.696372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df7b5b-b6d0-4306-adfc-e1a17eb61bf5 admin admin] PATH_INFO: `/v3/projects/8602e61d940c4578b8377741775dfe62/users/ef8a5e8073e942a29b6e33933b61ecf3/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:45.724590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df7b5b-b6d0-4306-adfc-e1a17eb61bf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8602e61d940c4578b8377741775dfe62, user_id=ef8a5e8073e942a29b6e33933b61ecf3, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:45.732476 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df7b5b-b6d0-4306-adfc-e1a17eb61bf5 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:45.749364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad2c9b-ec3d-49c5-9cb0-397c6a54aa57 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:45.755223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c7fbe-6ed7-4334-b313-e693ef6a9dbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:45.757838 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c7fbe-6ed7-4334-b313-e693ef6a9dbd service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:45.758507 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c7fbe-6ed7-4334-b313-e693ef6a9dbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:45.763519 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c7fbe-6ed7-4334-b313-e693ef6a9dbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:45.801092 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c7fbe-6ed7-4334-b313-e693ef6a9dbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:45.801092 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c7fbe-6ed7-4334-b313-e693ef6a9dbd service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:45.801724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1084/2171] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:45 2022] PUT /identity/v3/projects/8602e61d940c4578b8377741775dfe62/users/ef8a5e8073e942a29b6e33933b61ecf3/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:45.801724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679d0ba-5388-4fea-be1b-7c7cd90eb645 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:45.801724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679d0ba-5388-4fea-be1b-7c7cd90eb645 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:45.801724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679d0ba-5388-4fea-be1b-7c7cd90eb645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:45.812757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1088/2172] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:45.870486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-a679d0ba-5388-4fea-be1b-7c7cd90eb645 None None] MFA Rules not processed for user `ef8a5e8073e942a29b6e33933b61ecf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:45.871874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a679d0ba-5388-4fea-be1b-7c7cd90eb645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:45.969744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1085/2173] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:45 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:15:45.998671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c7fbe-6ed7-4334-b313-e693ef6a9dbd service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:46.000529 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ee2c7fbe-6ed7-4334-b313-e693ef6a9dbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:46.114184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c7fbe-6ed7-4334-b313-e693ef6a9dbd service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:46.268790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89f5b9-c5fb-43fa-bc70-e53347780bb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:46.270311 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89f5b9-c5fb-43fa-bc70-e53347780bb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:46.270619 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89f5b9-c5fb-43fa-bc70-e53347780bb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:46.271297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89f5b9-c5fb-43fa-bc70-e53347780bb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:46.344748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89f5b9-c5fb-43fa-bc70-e53347780bb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:46.344748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89f5b9-c5fb-43fa-bc70-e53347780bb0 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:46.373177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1089/2174] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:45 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:46.576642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679d0ba-5388-4fea-be1b-7c7cd90eb645 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:46.808087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679d0ba-5388-4fea-be1b-7c7cd90eb645 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:46.815804 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564be9ca-f134-477a-a942-6998c9b9f640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:46.817364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564be9ca-f134-477a-a942-6998c9b9f640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:46.817644 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564be9ca-f134-477a-a942-6998c9b9f640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:46.818050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564be9ca-f134-477a-a942-6998c9b9f640 admin admin] PATH_INFO: `/v3/users/09d64d2cc7e64e0497e3c571aa8d9c89` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:46.820071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564be9ca-f134-477a-a942-6998c9b9f640 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09d64d2cc7e64e0497e3c571aa8d9c89)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:46.822257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564be9ca-f134-477a-a942-6998c9b9f640 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:46.877087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:46.892287 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89f5b9-c5fb-43fa-bc70-e53347780bb0 service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:46.895767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3d89f5b9-c5fb-43fa-bc70-e53347780bb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:46.954119 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89f5b9-c5fb-43fa-bc70-e53347780bb0 service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:46.976406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc26b01-44de-49ce-8b1c-043bc4555872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:46.976406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc26b01-44de-49ce-8b1c-043bc4555872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:46.976406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc26b01-44de-49ce-8b1c-043bc4555872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:46.976406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc26b01-44de-49ce-8b1c-043bc4555872 admin admin] PATH_INFO: `/v3/projects/2fad77febdbf4b5d85253f80e566e89f` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:46.983712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc26b01-44de-49ce-8b1c-043bc4555872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fad77febdbf4b5d85253f80e566e89f)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:46.988560 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc26b01-44de-49ce-8b1c-043bc4555872 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:47.143348 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-564be9ca-f134-477a-a942-6998c9b9f640 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09d64d2cc7e64e0497e3c571aa8d9c89'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:47.144178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-564be9ca-f134-477a-a942-6998c9b9f640 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09d64d2cc7e64e0497e3c571aa8d9c89'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:47.145073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-564be9ca-f134-477a-a942-6998c9b9f640 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09d64d2cc7e64e0497e3c571aa8d9c89'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:47.168022 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:47.197298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1cc26b01-44de-49ce-8b1c-043bc4555872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fad77febdbf4b5d85253f80e566e89f', 'request_id': 'req-1cc26b01-44de-49ce-8b1c-043bc4555872'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:47.213144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-564be9ca-f134-477a-a942-6998c9b9f640 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09d64d2cc7e64e0497e3c571aa8d9c89'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:47.221435 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1cc26b01-44de-49ce-8b1c-043bc4555872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fad77febdbf4b5d85253f80e566e89f', 'request_id': 'req-1cc26b01-44de-49ce-8b1c-043bc4555872'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:47.229912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-564be9ca-f134-477a-a942-6998c9b9f640 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09d64d2cc7e64e0497e3c571aa8d9c89'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:47.247446 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1086/2175] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:15:46 2022] DELETE /identity/v3/users/09d64d2cc7e64e0497e3c571aa8d9c89 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:15:47.253316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7c1ec-c7f3-47ad-9e88-aa2c685617b3 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:47.253769 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7c1ec-c7f3-47ad-9e88-aa2c685617b3 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:47.255511 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7c1ec-c7f3-47ad-9e88-aa2c685617b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:47.280575 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-4df7c1ec-c7f3-47ad-9e88-aa2c685617b3 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:47.282711 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-4df7c1ec-c7f3-47ad-9e88-aa2c685617b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:47.342590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1cc26b01-44de-49ce-8b1c-043bc4555872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fad77febdbf4b5d85253f80e566e89f', 'request_id': 'req-1cc26b01-44de-49ce-8b1c-043bc4555872'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:47.362375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1cc26b01-44de-49ce-8b1c-043bc4555872 admin admin] The token cache is being invalidate because project 2fad77febdbf4b5d85253f80e566e89f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:15:47.364281 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1cc26b01-44de-49ce-8b1c-043bc4555872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:47.366764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1cc26b01-44de-49ce-8b1c-043bc4555872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:47.370247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1090/2176] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:15:46 2022] DELETE /identity/v3/projects/2fad77febdbf4b5d85253f80e566e89f => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:47.378085 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc26b01-44de-49ce-8b1c-043bc4555872 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:47.383989 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1cc26b01-44de-49ce-8b1c-043bc4555872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:47.431837 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1087/2177] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:47.440570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7c1ec-c7f3-47ad-9e88-aa2c685617b3 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:47.442877 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-4df7c1ec-c7f3-47ad-9e88-aa2c685617b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:47.639087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7c1ec-c7f3-47ad-9e88-aa2c685617b3 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:47.653297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f8356-66cb-4da8-a74d-e826a7475f2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:47.654130 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f8356-66cb-4da8-a74d-e826a7475f2f service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:47.655238 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f8356-66cb-4da8-a74d-e826a7475f2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:47.655238 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f8356-66cb-4da8-a74d-e826a7475f2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:47.668617 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc26b01-44de-49ce-8b1c-043bc4555872 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:47.669555 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f8356-66cb-4da8-a74d-e826a7475f2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:47.671658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f8356-66cb-4da8-a74d-e826a7475f2f service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:47.697451 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1088/2178] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:47.703481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f991ca7-2ff6-4cb7-9391-5fba709a6a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:47.704618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f991ca7-2ff6-4cb7-9391-5fba709a6a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:47.706452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f991ca7-2ff6-4cb7-9391-5fba709a6a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:47.706903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f991ca7-2ff6-4cb7-9391-5fba709a6a1c admin admin] PATH_INFO: `/v3/users/2606be1252c44a87a990598b6d63d6b2` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:47.710923 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f991ca7-2ff6-4cb7-9391-5fba709a6a1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2606be1252c44a87a990598b6d63d6b2)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:47.721493 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f991ca7-2ff6-4cb7-9391-5fba709a6a1c admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:48.012365 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6f991ca7-2ff6-4cb7-9391-5fba709a6a1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2606be1252c44a87a990598b6d63d6b2'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:48.028822 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6f991ca7-2ff6-4cb7-9391-5fba709a6a1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2606be1252c44a87a990598b6d63d6b2'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:48.031459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6f991ca7-2ff6-4cb7-9391-5fba709a6a1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2606be1252c44a87a990598b6d63d6b2'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:48.064520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6f991ca7-2ff6-4cb7-9391-5fba709a6a1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2606be1252c44a87a990598b6d63d6b2'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:48.200593 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6f991ca7-2ff6-4cb7-9391-5fba709a6a1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2606be1252c44a87a990598b6d63d6b2'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:48.208939 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1091/2179] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:15:47 2022] DELETE /identity/v3/users/2606be1252c44a87a990598b6d63d6b2 => generated 0 bytes in 835 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:15:48.349608 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:48.458404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:49.549176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:49.601519 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:49.638339 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f8356-66cb-4da8-a74d-e826a7475f2f service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:49.640117 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b81f8356-66cb-4da8-a74d-e826a7475f2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:49.667196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f8356-66cb-4da8-a74d-e826a7475f2f service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:49.823215 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:49.824539 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:49.825066 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:49.825571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94 admin admin] PATH_INFO: `/v3/projects/305c3698fba14a46b027e557fb68d7d7` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:49.827284 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=305c3698fba14a46b027e557fb68d7d7)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:49.831718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:50.074930 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '305c3698fba14a46b027e557fb68d7d7', 'request_id': 'req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:50.088918 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '305c3698fba14a46b027e557fb68d7d7', 'request_id': 'req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:50.093124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f991ca7-2ff6-4cb7-9391-5fba709a6a1c admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:50.094786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-6f991ca7-2ff6-4cb7-9391-5fba709a6a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:50.116306 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '305c3698fba14a46b027e557fb68d7d7', 'request_id': 'req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:50.125805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f991ca7-2ff6-4cb7-9391-5fba709a6a1c admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:50.252743 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb1f32-abcb-40c5-86b6-95d4d0186704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:50.253655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb1f32-abcb-40c5-86b6-95d4d0186704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:50.253939 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb1f32-abcb-40c5-86b6-95d4d0186704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:50.254217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb1f32-abcb-40c5-86b6-95d4d0186704 admin admin] PATH_INFO: `/v3/projects/3b75c9c598704bffa405dc9be4ffa9b9` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:50.255659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb1f32-abcb-40c5-86b6-95d4d0186704 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b75c9c598704bffa405dc9be4ffa9b9)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:50.259077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb1f32-abcb-40c5-86b6-95d4d0186704 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:50.285999 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94 admin admin] The token cache is being invalidate because project 305c3698fba14a46b027e557fb68d7d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:15:50.286422 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:50.287207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:50.292726 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1089/2180] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:15:49 2022] DELETE /identity/v3/projects/305c3698fba14a46b027e557fb68d7d7 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:50.293781 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:50.295354 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:50.328601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa1440-9c4f-4bfe-becf-8357ee6cfc94 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:50.334728 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f42985-ce3a-49ea-896d-e8f4077ac52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:50.335747 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f42985-ce3a-49ea-896d-e8f4077ac52b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:50.336353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f42985-ce3a-49ea-896d-e8f4077ac52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:50.336811 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f42985-ce3a-49ea-896d-e8f4077ac52b admin admin] PATH_INFO: `/v3/users/ef8a5e8073e942a29b6e33933b61ecf3` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:50.339508 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f42985-ce3a-49ea-896d-e8f4077ac52b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef8a5e8073e942a29b6e33933b61ecf3)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:50.341674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f42985-ce3a-49ea-896d-e8f4077ac52b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:50.541631 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-26eb1f32-abcb-40c5-86b6-95d4d0186704 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b75c9c598704bffa405dc9be4ffa9b9', 'request_id': 'req-26eb1f32-abcb-40c5-86b6-95d4d0186704'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:50.547124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-26eb1f32-abcb-40c5-86b6-95d4d0186704 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b75c9c598704bffa405dc9be4ffa9b9', 'request_id': 'req-26eb1f32-abcb-40c5-86b6-95d4d0186704'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:50.664183 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-26eb1f32-abcb-40c5-86b6-95d4d0186704 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b75c9c598704bffa405dc9be4ffa9b9', 'request_id': 'req-26eb1f32-abcb-40c5-86b6-95d4d0186704'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:50.684950 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-26eb1f32-abcb-40c5-86b6-95d4d0186704 admin admin] The token cache is being invalidate because project 3b75c9c598704bffa405dc9be4ffa9b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:15:50.686431 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-26eb1f32-abcb-40c5-86b6-95d4d0186704 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:50.687416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-26eb1f32-abcb-40c5-86b6-95d4d0186704 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:50.695359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1092/2181] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:15:50 2022] DELETE /identity/v3/projects/3b75c9c598704bffa405dc9be4ffa9b9 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:50.696873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb1f32-abcb-40c5-86b6-95d4d0186704 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:50.698406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-26eb1f32-abcb-40c5-86b6-95d4d0186704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:50.708110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a1f42985-ce3a-49ea-896d-e8f4077ac52b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef8a5e8073e942a29b6e33933b61ecf3'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:50.710994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a1f42985-ce3a-49ea-896d-e8f4077ac52b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef8a5e8073e942a29b6e33933b61ecf3'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:50.712365 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a1f42985-ce3a-49ea-896d-e8f4077ac52b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef8a5e8073e942a29b6e33933b61ecf3'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:50.817720 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a1f42985-ce3a-49ea-896d-e8f4077ac52b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef8a5e8073e942a29b6e33933b61ecf3'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:50.840733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:50.860739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a1f42985-ce3a-49ea-896d-e8f4077ac52b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef8a5e8073e942a29b6e33933b61ecf3'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:50.877761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb1f32-abcb-40c5-86b6-95d4d0186704 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:50.899126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b973ff1-2604-47e9-a8e4-3719a43134c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:50.900281 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b973ff1-2604-47e9-a8e4-3719a43134c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:50.900571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b973ff1-2604-47e9-a8e4-3719a43134c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:50.900854 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b973ff1-2604-47e9-a8e4-3719a43134c0 admin admin] PATH_INFO: `/v3/users/b195ad7f5e074ec3af0da9d80f6854ff` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:50.908654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b973ff1-2604-47e9-a8e4-3719a43134c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b195ad7f5e074ec3af0da9d80f6854ff)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:50.912564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b973ff1-2604-47e9-a8e4-3719a43134c0 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:50.919163 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1090/2182] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:15:50 2022] DELETE /identity/v3/users/ef8a5e8073e942a29b6e33933b61ecf3 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:50.922403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:51.339837 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2b973ff1-2604-47e9-a8e4-3719a43134c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b195ad7f5e074ec3af0da9d80f6854ff'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:51.377869 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2b973ff1-2604-47e9-a8e4-3719a43134c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b195ad7f5e074ec3af0da9d80f6854ff'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:51.381885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2b973ff1-2604-47e9-a8e4-3719a43134c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b195ad7f5e074ec3af0da9d80f6854ff'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:51.422501 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2b973ff1-2604-47e9-a8e4-3719a43134c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b195ad7f5e074ec3af0da9d80f6854ff'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:52.319928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2b973ff1-2604-47e9-a8e4-3719a43134c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b195ad7f5e074ec3af0da9d80f6854ff'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:52.328797 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1093/2183] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:15:50 2022] DELETE /identity/v3/users/b195ad7f5e074ec3af0da9d80f6854ff => generated 0 bytes in 1632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:52.622578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f42985-ce3a-49ea-896d-e8f4077ac52b admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:52.625651 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a1f42985-ce3a-49ea-896d-e8f4077ac52b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:52.770249 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f42985-ce3a-49ea-896d-e8f4077ac52b admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:52.871903 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f787a-2b24-41ae-9aa8-717f34ac5a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:52.871903 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f787a-2b24-41ae-9aa8-717f34ac5a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:52.871903 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f787a-2b24-41ae-9aa8-717f34ac5a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:52.871903 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f787a-2b24-41ae-9aa8-717f34ac5a84 admin admin] PATH_INFO: `/v3/projects/8602e61d940c4578b8377741775dfe62` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:52.872279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f787a-2b24-41ae-9aa8-717f34ac5a84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8602e61d940c4578b8377741775dfe62)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:52.874890 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f787a-2b24-41ae-9aa8-717f34ac5a84 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:53.029101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ee3f787a-2b24-41ae-9aa8-717f34ac5a84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8602e61d940c4578b8377741775dfe62', 'request_id': 'req-ee3f787a-2b24-41ae-9aa8-717f34ac5a84'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:53.035499 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ee3f787a-2b24-41ae-9aa8-717f34ac5a84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8602e61d940c4578b8377741775dfe62', 'request_id': 'req-ee3f787a-2b24-41ae-9aa8-717f34ac5a84'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:53.044848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ee3f787a-2b24-41ae-9aa8-717f34ac5a84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8602e61d940c4578b8377741775dfe62', 'request_id': 'req-ee3f787a-2b24-41ae-9aa8-717f34ac5a84'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:53.106078 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ee3f787a-2b24-41ae-9aa8-717f34ac5a84 admin admin] The token cache is being invalidate because project 8602e61d940c4578b8377741775dfe62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:15:53.106474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ee3f787a-2b24-41ae-9aa8-717f34ac5a84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:53.108423 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ee3f787a-2b24-41ae-9aa8-717f34ac5a84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:53.112146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1091/2184] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:15:52 2022] DELETE /identity/v3/projects/8602e61d940c4578b8377741775dfe62 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:53.121079 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b5fc7-f7b9-4927-87ab-46a9760d4580 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:53.121397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b5fc7-f7b9-4927-87ab-46a9760d4580 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:53.121975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b5fc7-f7b9-4927-87ab-46a9760d4580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:53.150728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-502b5fc7-f7b9-4927-87ab-46a9760d4580 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:53.152412 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-502b5fc7-f7b9-4927-87ab-46a9760d4580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:53.279920 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1094/2185] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:53.282326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980901a-ce59-4ae0-b529-9c7176f972e4 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:53.282326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980901a-ce59-4ae0-b529-9c7176f972e4 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:53.282326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980901a-ce59-4ae0-b529-9c7176f972e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:53.322875 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-c980901a-ce59-4ae0-b529-9c7176f972e4 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:53.345729 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c980901a-ce59-4ae0-b529-9c7176f972e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:53.345729 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1092/2186] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:53.352543 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b5fc7-f7b9-4927-87ab-46a9760d4580 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:53.377647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b5fc7-f7b9-4927-87ab-46a9760d4580 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:53.394830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcbc6b5-5ec7-40d4-809d-644ccf33a786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:53.396857 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbc6b5-5ec7-40d4-809d-644ccf33a786 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:53.397978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbc6b5-5ec7-40d4-809d-644ccf33a786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:53.399436 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbc6b5-5ec7-40d4-809d-644ccf33a786 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:53.416894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbc6b5-5ec7-40d4-809d-644ccf33a786 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:15:53.416894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbc6b5-5ec7-40d4-809d-644ccf33a786 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:53.416894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbc6b5-5ec7-40d4-809d-644ccf33a786 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:53.422906 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1095/2187] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 15:15:53.427364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980901a-ce59-4ae0-b529-9c7176f972e4 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:53.443432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980901a-ce59-4ae0-b529-9c7176f972e4 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:53.460228 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5546faa-99c4-458d-8a42-afa549ba6063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:53.460228 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5546faa-99c4-458d-8a42-afa549ba6063 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:53.461028 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5546faa-99c4-458d-8a42-afa549ba6063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:53.461393 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5546faa-99c4-458d-8a42-afa549ba6063 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:53.464362 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5546faa-99c4-458d-8a42-afa549ba6063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:53.465255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5546faa-99c4-458d-8a42-afa549ba6063 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:53.519910 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcbc6b5-5ec7-40d4-809d-644ccf33a786 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:53.524785 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3fcbc6b5-5ec7-40d4-809d-644ccf33a786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:53.540846 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcbc6b5-5ec7-40d4-809d-644ccf33a786 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:53.550973 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392125e-056b-40db-8c66-d51129dd8159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:53.553006 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392125e-056b-40db-8c66-d51129dd8159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:53.554222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392125e-056b-40db-8c66-d51129dd8159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:53.556837 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392125e-056b-40db-8c66-d51129dd8159 admin admin] PATH_INFO: `/v3/projects/edb7dad9b7824254988eb995745aa879` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:53.559730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392125e-056b-40db-8c66-d51129dd8159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edb7dad9b7824254988eb995745aa879)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:53.561961 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392125e-056b-40db-8c66-d51129dd8159 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:53.625091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1093/2188] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:15:53 2022] POST /identity/v3/projects => generated 367 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:53.628667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5546faa-99c4-458d-8a42-afa549ba6063 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:53.778774 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5546faa-99c4-458d-8a42-afa549ba6063 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:53.807638 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2392125e-056b-40db-8c66-d51129dd8159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edb7dad9b7824254988eb995745aa879', 'request_id': 'req-2392125e-056b-40db-8c66-d51129dd8159'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:53.817959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2392125e-056b-40db-8c66-d51129dd8159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edb7dad9b7824254988eb995745aa879', 'request_id': 'req-2392125e-056b-40db-8c66-d51129dd8159'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:53.915561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2392125e-056b-40db-8c66-d51129dd8159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edb7dad9b7824254988eb995745aa879', 'request_id': 'req-2392125e-056b-40db-8c66-d51129dd8159'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:53.935917 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587101c-e435-4b96-aa33-c36e0049daff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:53.937954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587101c-e435-4b96-aa33-c36e0049daff admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:53.938266 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587101c-e435-4b96-aa33-c36e0049daff admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:53.938584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587101c-e435-4b96-aa33-c36e0049daff admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:53.939667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587101c-e435-4b96-aa33-c36e0049daff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:53.942044 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587101c-e435-4b96-aa33-c36e0049daff admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:53.945762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2392125e-056b-40db-8c66-d51129dd8159 admin admin] The token cache is being invalidate because project edb7dad9b7824254988eb995745aa879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:15:53.946851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2392125e-056b-40db-8c66-d51129dd8159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:53.946851 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2392125e-056b-40db-8c66-d51129dd8159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:15:53.949133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1096/2189] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:15:53 2022] DELETE /identity/v3/projects/edb7dad9b7824254988eb995745aa879 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:53.955175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451ae78-0539-44a0-a84b-c8265df3cc47 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:53.955676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451ae78-0539-44a0-a84b-c8265df3cc47 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:53.956387 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451ae78-0539-44a0-a84b-c8265df3cc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:53.962753 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:54.003979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-c451ae78-0539-44a0-a84b-c8265df3cc47 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:54.007294 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c451ae78-0539-44a0-a84b-c8265df3cc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:54.020025 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1097/2190] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:54.032216 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4278234-a586-42f5-9b51-02a09b9c45c5 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:54.032432 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4278234-a586-42f5-9b51-02a09b9c45c5 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:54.032713 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4278234-a586-42f5-9b51-02a09b9c45c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:54.086865 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1094/2191] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:15:53 2022] POST /identity/v3/users => generated 284 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:54.092100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587101c-e435-4b96-aa33-c36e0049daff admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:54.096730 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6587101c-e435-4b96-aa33-c36e0049daff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:54.137460 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587101c-e435-4b96-aa33-c36e0049daff admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:54.138438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-f4278234-a586-42f5-9b51-02a09b9c45c5 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:54.140355 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f4278234-a586-42f5-9b51-02a09b9c45c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:54.144990 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798e929-40e0-4ebe-a81e-23027b4ba16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:54.146171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798e929-40e0-4ebe-a81e-23027b4ba16b admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:54.146716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798e929-40e0-4ebe-a81e-23027b4ba16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:54.151241 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798e929-40e0-4ebe-a81e-23027b4ba16b admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:54.151241 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798e929-40e0-4ebe-a81e-23027b4ba16b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:54.152987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798e929-40e0-4ebe-a81e-23027b4ba16b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:54.165959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1098/2192] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:54.167965 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1095/2193] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:54.170249 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4278234-a586-42f5-9b51-02a09b9c45c5 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:54.172470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798e929-40e0-4ebe-a81e-23027b4ba16b admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:54.188647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798e929-40e0-4ebe-a81e-23027b4ba16b admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:54.194013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52145aa2-d013-4823-9af8-201e20a8f441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:54.194918 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52145aa2-d013-4823-9af8-201e20a8f441 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:54.195322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52145aa2-d013-4823-9af8-201e20a8f441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:54.195613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52145aa2-d013-4823-9af8-201e20a8f441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:54.196374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52145aa2-d013-4823-9af8-201e20a8f441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:54.197676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4278234-a586-42f5-9b51-02a09b9c45c5 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:54.198003 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52145aa2-d013-4823-9af8-201e20a8f441 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:54.208633 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39989045-418d-42ba-940c-a33bde32a762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:54.211890 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39989045-418d-42ba-940c-a33bde32a762 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:54.212180 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39989045-418d-42ba-940c-a33bde32a762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:54.212465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39989045-418d-42ba-940c-a33bde32a762 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:54.213484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39989045-418d-42ba-940c-a33bde32a762 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:15:54.213795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39989045-418d-42ba-940c-a33bde32a762 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:54.220117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39989045-418d-42ba-940c-a33bde32a762 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:54.232487 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1096/2194] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:54.239314 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52145aa2-d013-4823-9af8-201e20a8f441 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:54.252459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52145aa2-d013-4823-9af8-201e20a8f441 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:54.255715 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1099/2195] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 15:15:54.261115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d102ea9-a57d-4db3-9a65-b0667b68d4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:54.262341 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d102ea9-a57d-4db3-9a65-b0667b68d4ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:54.262951 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d102ea9-a57d-4db3-9a65-b0667b68d4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:54.263429 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d102ea9-a57d-4db3-9a65-b0667b68d4ef admin admin] PATH_INFO: `/v3/projects/1ed01e2b97b7457a857452918a166012/users/72b7d968e5dd4d4d8c6ca2ee313264d6/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:54.265742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39989045-418d-42ba-940c-a33bde32a762 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:54.279070 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39989045-418d-42ba-940c-a33bde32a762 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:54.283465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d102ea9-a57d-4db3-9a65-b0667b68d4ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ed01e2b97b7457a857452918a166012, user_id=72b7d968e5dd4d4d8c6ca2ee313264d6, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:54.298494 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d102ea9-a57d-4db3-9a65-b0667b68d4ef admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:54.305236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3121de-c43b-40ed-a9e0-b5b4635aa097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:54.305776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3121de-c43b-40ed-a9e0-b5b4635aa097 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:54.306066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3121de-c43b-40ed-a9e0-b5b4635aa097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:54.306325 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3121de-c43b-40ed-a9e0-b5b4635aa097 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:54.308208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3121de-c43b-40ed-a9e0-b5b4635aa097 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:54.317940 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3121de-c43b-40ed-a9e0-b5b4635aa097 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:54.393096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1097/2196] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:54 2022] PUT /identity/v3/projects/1ed01e2b97b7457a857452918a166012/users/72b7d968e5dd4d4d8c6ca2ee313264d6/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:54.408743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb11a1-657f-4f48-b871-77b3e6c4a595 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:54.410262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb11a1-657f-4f48-b871-77b3e6c4a595 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:54.410914 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb11a1-657f-4f48-b871-77b3e6c4a595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:54.461693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1100/2197] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:15:54 2022] POST /identity/v3/projects => generated 383 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:54.466737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3121de-c43b-40ed-a9e0-b5b4635aa097 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:54.492449 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-9bdb11a1-657f-4f48-b871-77b3e6c4a595 None None] MFA Rules not processed for user `72b7d968e5dd4d4d8c6ca2ee313264d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:54.499482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9bdb11a1-657f-4f48-b871-77b3e6c4a595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:54.554220 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:54.660236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3121de-c43b-40ed-a9e0-b5b4635aa097 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:54.671061 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6aa53-a97c-4fcb-a0bd-71423e571e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:54.671885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6aa53-a97c-4fcb-a0bd-71423e571e71 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:54.672281 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6aa53-a97c-4fcb-a0bd-71423e571e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:54.672666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6aa53-a97c-4fcb-a0bd-71423e571e71 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:54.678962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6aa53-a97c-4fcb-a0bd-71423e571e71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:54.681639 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6aa53-a97c-4fcb-a0bd-71423e571e71 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:54.793897 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1098/2198] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:54 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:54.801110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb11a1-657f-4f48-b871-77b3e6c4a595 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:54.815925 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1101/2199] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:15:54 2022] POST /identity/v3/users => generated 292 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:54.817192 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6aa53-a97c-4fcb-a0bd-71423e571e71 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:54.829250 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb11a1-657f-4f48-b871-77b3e6c4a595 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:54.836330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c1d5b-8888-4f86-940e-62248080d5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:54.836618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c1d5b-8888-4f86-940e-62248080d5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:54.836740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c1d5b-8888-4f86-940e-62248080d5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:54.837022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c1d5b-8888-4f86-940e-62248080d5c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:54.837806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c1d5b-8888-4f86-940e-62248080d5c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:54.839672 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6aa53-a97c-4fcb-a0bd-71423e571e71 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:54.840189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c1d5b-8888-4f86-940e-62248080d5c3 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:54.861525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51aed48-db25-442e-a87a-3447986c1fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:54.862738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51aed48-db25-442e-a87a-3447986c1fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:54.863121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51aed48-db25-442e-a87a-3447986c1fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:54.875116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51aed48-db25-442e-a87a-3447986c1fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:54.875116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51aed48-db25-442e-a87a-3447986c1fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:54.875116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51aed48-db25-442e-a87a-3447986c1fd8 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:54.896715 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1102/2200] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:54.903523 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51aed48-db25-442e-a87a-3447986c1fd8 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:54.920534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51aed48-db25-442e-a87a-3447986c1fd8 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:54.939667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1099/2201] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:15:54 2022] POST /identity/v3/projects => generated 367 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:54.940597 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c1d5b-8888-4f86-940e-62248080d5c3 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:54.946081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ff647-f94a-4f19-880b-919e4674964b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:54.946445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ff647-f94a-4f19-880b-919e4674964b admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:54.946445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ff647-f94a-4f19-880b-919e4674964b admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:54.946809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ff647-f94a-4f19-880b-919e4674964b admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:54.949748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ff647-f94a-4f19-880b-919e4674964b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:54.951509 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ff647-f94a-4f19-880b-919e4674964b admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:54.979302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1103/2202] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:54.980282 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ff647-f94a-4f19-880b-919e4674964b admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:55.011994 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ff647-f94a-4f19-880b-919e4674964b admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:55.142199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c1d5b-8888-4f86-940e-62248080d5c3 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:55.157809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10002f74-60fb-413e-8664-fea0247aa574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:55.158578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10002f74-60fb-413e-8664-fea0247aa574 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:55.158861 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10002f74-60fb-413e-8664-fea0247aa574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:55.159158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10002f74-60fb-413e-8664-fea0247aa574 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:55.159932 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10002f74-60fb-413e-8664-fea0247aa574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:55.161990 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10002f74-60fb-413e-8664-fea0247aa574 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:55.224571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae52ee-85f9-4c89-a75a-128dbec6061f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:55.230489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae52ee-85f9-4c89-a75a-128dbec6061f admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:55.230489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae52ee-85f9-4c89-a75a-128dbec6061f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:55.230489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae52ee-85f9-4c89-a75a-128dbec6061f admin admin] PATH_INFO: `/v3/projects/860223baafd746d8a515d3d6082c3e88/users/b95ae3519ae74d4b92175f14187c01f7/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:55.251775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1100/2203] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:15:54 2022] POST /identity/v3/users => generated 290 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:15:55.253210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10002f74-60fb-413e-8664-fea0247aa574 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:55.271852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae52ee-85f9-4c89-a75a-128dbec6061f admin admin] RBAC: Authorizing `identity:create_grant(project_id=860223baafd746d8a515d3d6082c3e88, user_id=b95ae3519ae74d4b92175f14187c01f7, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:55.282855 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae52ee-85f9-4c89-a75a-128dbec6061f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:55.288170 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10002f74-60fb-413e-8664-fea0247aa574 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:55.298353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c7098-f709-42ad-8987-0ee38ce59c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:55.302805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c7098-f709-42ad-8987-0ee38ce59c47 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:55.304025 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c7098-f709-42ad-8987-0ee38ce59c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:55.304547 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c7098-f709-42ad-8987-0ee38ce59c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:55.307273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c7098-f709-42ad-8987-0ee38ce59c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:55.316747 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c7098-f709-42ad-8987-0ee38ce59c47 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:55.361537 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1101/2204] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:55.362921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c7098-f709-42ad-8987-0ee38ce59c47 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:55.377265 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c7098-f709-42ad-8987-0ee38ce59c47 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:55.381178 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1104/2205] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:54 2022] PUT /identity/v3/projects/860223baafd746d8a515d3d6082c3e88/users/b95ae3519ae74d4b92175f14187c01f7/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:55.392407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acc7a6-ab0c-42b0-9542-c62e498d4cc3 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:55.392953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acc7a6-ab0c-42b0-9542-c62e498d4cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:55.393250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acc7a6-ab0c-42b0-9542-c62e498d4cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:55.488755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-81acc7a6-ab0c-42b0-9542-c62e498d4cc3 None None] MFA Rules not processed for user `b95ae3519ae74d4b92175f14187c01f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:55.495878 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-81acc7a6-ab0c-42b0-9542-c62e498d4cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:55.565232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7920735-c414-443d-aa4f-5d0f1caadd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:55.566410 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7920735-c414-443d-aa4f-5d0f1caadd30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:55.567761 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7920735-c414-443d-aa4f-5d0f1caadd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:55.567761 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7920735-c414-443d-aa4f-5d0f1caadd30 admin admin] PATH_INFO: `/v3/projects/ea7e31f83f8540b6ba4670c7f38c38eb/users/ddb83c487e51468281c64892faf091f6/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:55.631063 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7920735-c414-443d-aa4f-5d0f1caadd30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea7e31f83f8540b6ba4670c7f38c38eb, user_id=ddb83c487e51468281c64892faf091f6, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:55.646588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7920735-c414-443d-aa4f-5d0f1caadd30 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:55.773902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1102/2206] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:55 2022] PUT /identity/v3/projects/ea7e31f83f8540b6ba4670c7f38c38eb/users/ddb83c487e51468281c64892faf091f6/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:55.775956 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7920735-c414-443d-aa4f-5d0f1caadd30 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:55.793704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1105/2207] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:15:55.795413 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acc7a6-ab0c-42b0-9542-c62e498d4cc3 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:55.839406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acc7a6-ab0c-42b0-9542-c62e498d4cc3 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:55.911002 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7920735-c414-443d-aa4f-5d0f1caadd30 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:55.932862 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e463818-2172-4a42-baae-bf46dab6f213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:55.934923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e463818-2172-4a42-baae-bf46dab6f213 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:55.935269 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e463818-2172-4a42-baae-bf46dab6f213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:55.935508 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e463818-2172-4a42-baae-bf46dab6f213 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:55.938416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e463818-2172-4a42-baae-bf46dab6f213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:55.940248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e463818-2172-4a42-baae-bf46dab6f213 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:55.954438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1103/2208] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:55.958754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e463818-2172-4a42-baae-bf46dab6f213 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:55.970020 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e463818-2172-4a42-baae-bf46dab6f213 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:55.977091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41673977-1c1d-4ebb-a5cd-8756fcca33c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:55.978362 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41673977-1c1d-4ebb-a5cd-8756fcca33c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:55.978362 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41673977-1c1d-4ebb-a5cd-8756fcca33c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:55.978738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41673977-1c1d-4ebb-a5cd-8756fcca33c6 admin admin] PATH_INFO: `/v3/projects/ea7e31f83f8540b6ba4670c7f38c38eb/users/ddb83c487e51468281c64892faf091f6/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:55.994820 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41673977-1c1d-4ebb-a5cd-8756fcca33c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea7e31f83f8540b6ba4670c7f38c38eb, user_id=ddb83c487e51468281c64892faf091f6, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:55.996754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15c734-7bf7-4042-ac97-015eb8f3eb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:55.997710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15c734-7bf7-4042-ac97-015eb8f3eb0f admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:55.998235 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15c734-7bf7-4042-ac97-015eb8f3eb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:55.998768 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15c734-7bf7-4042-ac97-015eb8f3eb0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:55.999686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15c734-7bf7-4042-ac97-015eb8f3eb0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:56.002111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15c734-7bf7-4042-ac97-015eb8f3eb0f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:56.010732 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41673977-1c1d-4ebb-a5cd-8756fcca33c6 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:56.041734 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1104/2209] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:55 2022] PUT /identity/v3/projects/ea7e31f83f8540b6ba4670c7f38c38eb/users/ddb83c487e51468281c64892faf091f6/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:56.057021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad08d6-2885-4eaa-bf2c-d58d7addb267 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:56.057021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad08d6-2885-4eaa-bf2c-d58d7addb267 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:56.057021 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad08d6-2885-4eaa-bf2c-d58d7addb267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:56.093118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:56.104209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-92ad08d6-2885-4eaa-bf2c-d58d7addb267 None None] MFA Rules not processed for user `ddb83c487e51468281c64892faf091f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:56.106761 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-92ad08d6-2885-4eaa-bf2c-d58d7addb267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:56.149650 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1106/2210] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:15:55 2022] POST /identity/v3/projects => generated 383 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:56.150904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15c734-7bf7-4042-ac97-015eb8f3eb0f admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:56.308665 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1105/2211] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:56 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:56.326859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad08d6-2885-4eaa-bf2c-d58d7addb267 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:56.332895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-92ad08d6-2885-4eaa-bf2c-d58d7addb267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:56.334024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15c734-7bf7-4042-ac97-015eb8f3eb0f admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:56.342084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd9b67-8334-42d7-a48e-f9c2da741239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:56.343057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd9b67-8334-42d7-a48e-f9c2da741239 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:56.343604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd9b67-8334-42d7-a48e-f9c2da741239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:56.343882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd9b67-8334-42d7-a48e-f9c2da741239 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:56.344896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd9b67-8334-42d7-a48e-f9c2da741239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:56.347302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd9b67-8334-42d7-a48e-f9c2da741239 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:56.443808 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad08d6-2885-4eaa-bf2c-d58d7addb267 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:56.456305 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1107/2212] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:15:56 2022] POST /identity/v3/users => generated 298 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:15:56.461041 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd9b67-8334-42d7-a48e-f9c2da741239 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:56.524353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd9b67-8334-42d7-a48e-f9c2da741239 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:56.543210 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff0fa5-0b6d-4dcb-85dd-7659df1bd09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:56.544003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff0fa5-0b6d-4dcb-85dd-7659df1bd09d admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:56.544182 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff0fa5-0b6d-4dcb-85dd-7659df1bd09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:56.544449 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff0fa5-0b6d-4dcb-85dd-7659df1bd09d admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:56.551826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff0fa5-0b6d-4dcb-85dd-7659df1bd09d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:56.556523 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff0fa5-0b6d-4dcb-85dd-7659df1bd09d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:56.593397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1108/2213] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:15:56.596505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff0fa5-0b6d-4dcb-85dd-7659df1bd09d admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:56.623188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff0fa5-0b6d-4dcb-85dd-7659df1bd09d admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:56.636102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5766a8-eb16-4875-b84c-a5264b71f9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:56.636873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5766a8-eb16-4875-b84c-a5264b71f9bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:56.637200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5766a8-eb16-4875-b84c-a5264b71f9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:56.637481 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5766a8-eb16-4875-b84c-a5264b71f9bf admin admin] PATH_INFO: `/v3/projects/84b530ae96604fa4a55f9be915e9d503/users/9f3c597f6bf24d7ba5d46b53b71e8219/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:56.652339 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c30256-84f1-46fb-9f2b-d77b6c255f23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:56.653373 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c30256-84f1-46fb-9f2b-d77b6c255f23 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:56.653840 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c30256-84f1-46fb-9f2b-d77b6c255f23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:56.654289 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c30256-84f1-46fb-9f2b-d77b6c255f23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:56.676569 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5766a8-eb16-4875-b84c-a5264b71f9bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b530ae96604fa4a55f9be915e9d503, user_id=9f3c597f6bf24d7ba5d46b53b71e8219, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:56.682527 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5766a8-eb16-4875-b84c-a5264b71f9bf admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:56.703676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:56.776382 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1109/2214] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:56 2022] PUT /identity/v3/projects/84b530ae96604fa4a55f9be915e9d503/users/9f3c597f6bf24d7ba5d46b53b71e8219/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:15:56.780709 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5766a8-eb16-4875-b84c-a5264b71f9bf admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:56.848190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c30256-84f1-46fb-9f2b-d77b6c255f23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:56.849828 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c30256-84f1-46fb-9f2b-d77b6c255f23 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:56.864675 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1106/2215] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:56 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:56.949187 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5766a8-eb16-4875-b84c-a5264b71f9bf admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:56.963738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d33502-0519-4027-a14d-ddc5bbb3e52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:56.964477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d33502-0519-4027-a14d-ddc5bbb3e52f admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:56.964820 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d33502-0519-4027-a14d-ddc5bbb3e52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:56.967305 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d33502-0519-4027-a14d-ddc5bbb3e52f admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:56.968095 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d33502-0519-4027-a14d-ddc5bbb3e52f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:56.973239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d33502-0519-4027-a14d-ddc5bbb3e52f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:57.003638 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c30256-84f1-46fb-9f2b-d77b6c255f23 service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:57.004521 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1110/2216] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:15:57.040391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c30256-84f1-46fb-9f2b-d77b6c255f23 service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:57.067590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2309694-b6c3-4012-9518-ec932890475a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:57.069456 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2309694-b6c3-4012-9518-ec932890475a admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:57.069763 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2309694-b6c3-4012-9518-ec932890475a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:57.073010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2309694-b6c3-4012-9518-ec932890475a admin admin] PATH_INFO: `/v3/projects/84b530ae96604fa4a55f9be915e9d503/users/9f3c597f6bf24d7ba5d46b53b71e8219/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:57.077895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2309694-b6c3-4012-9518-ec932890475a admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b530ae96604fa4a55f9be915e9d503, user_id=9f3c597f6bf24d7ba5d46b53b71e8219, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:57.086150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2309694-b6c3-4012-9518-ec932890475a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:57.121701 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1107/2217] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:15:57 2022] PUT /identity/v3/projects/84b530ae96604fa4a55f9be915e9d503/users/9f3c597f6bf24d7ba5d46b53b71e8219/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:15:57.135721 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2660c1a-1397-4b90-a90e-0f7b2a4313ed None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:57.136065 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2660c1a-1397-4b90-a90e-0f7b2a4313ed None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:57.136293 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2660c1a-1397-4b90-a90e-0f7b2a4313ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:57.205790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-b2660c1a-1397-4b90-a90e-0f7b2a4313ed None None] MFA Rules not processed for user `9f3c597f6bf24d7ba5d46b53b71e8219`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:15:57.207115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b2660c1a-1397-4b90-a90e-0f7b2a4313ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:15:57.416832 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1111/2218] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:15:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:15:57.427096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2309694-b6c3-4012-9518-ec932890475a admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:57.598403 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2309694-b6c3-4012-9518-ec932890475a admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:57.609737 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ee2b2-c846-4112-acb6-3bf2c050963d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:57.611259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ee2b2-c846-4112-acb6-3bf2c050963d service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:57.611569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ee2b2-c846-4112-acb6-3bf2c050963d service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:57.611858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ee2b2-c846-4112-acb6-3bf2c050963d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:57.649190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ee2b2-c846-4112-acb6-3bf2c050963d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:57.655986 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ee2b2-c846-4112-acb6-3bf2c050963d service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:57.701644 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1108/2219] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:57.876901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2660c1a-1397-4b90-a90e-0f7b2a4313ed None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:15:57.890002 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2660c1a-1397-4b90-a90e-0f7b2a4313ed None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:15:57.909685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cce72f-9596-47a3-930d-76dea7fe907e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:15:57.909685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cce72f-9596-47a3-930d-76dea7fe907e service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:15:57.909685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cce72f-9596-47a3-930d-76dea7fe907e service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:15:57.909685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cce72f-9596-47a3-930d-76dea7fe907e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:15:58.093113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cce72f-9596-47a3-930d-76dea7fe907e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:15:58.099632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cce72f-9596-47a3-930d-76dea7fe907e service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:15:58.133066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1112/2220] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:15:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:15:58.493075 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:15:58.933436 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:00.704833 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:01.598572 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75338a-d7e0-4575-a689-daea84e4c545 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:01.606997 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75338a-d7e0-4575-a689-daea84e4c545 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:01.607331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75338a-d7e0-4575-a689-daea84e4c545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:01.667004 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-3d75338a-d7e0-4575-a689-daea84e4c545 None None] MFA Rules not processed for user `dcfdd71f9b93426bb6b149d2aeb2088e`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:01.682077 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-3d75338a-d7e0-4575-a689-daea84e4c545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:01.797064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cce72f-9596-47a3-930d-76dea7fe907e service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:01.802576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1109/2221] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:01.805148 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75338a-d7e0-4575-a689-daea84e4c545 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:01.807661 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-3d75338a-d7e0-4575-a689-daea84e4c545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:01.927739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75338a-d7e0-4575-a689-daea84e4c545 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:01.941704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cce72f-9596-47a3-930d-76dea7fe907e service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:01.942784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65714fc-0dd6-4b35-9edc-ea4463c1963f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:01.947438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65714fc-0dd6-4b35-9edc-ea4463c1963f admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:01.947438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65714fc-0dd6-4b35-9edc-ea4463c1963f admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:01.947438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65714fc-0dd6-4b35-9edc-ea4463c1963f admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:01.947438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65714fc-0dd6-4b35-9edc-ea4463c1963f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:01.951823 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d3d92-8ff4-4bee-b2c3-72fe2a2ed902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:01.952224 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65714fc-0dd6-4b35-9edc-ea4463c1963f admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:01.953197 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d3d92-8ff4-4bee-b2c3-72fe2a2ed902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:01.953514 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d3d92-8ff4-4bee-b2c3-72fe2a2ed902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:01.953870 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d3d92-8ff4-4bee-b2c3-72fe2a2ed902 admin admin] PATH_INFO: `/v3/users/72b7d968e5dd4d4d8c6ca2ee313264d6` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:01.955557 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d3d92-8ff4-4bee-b2c3-72fe2a2ed902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72b7d968e5dd4d4d8c6ca2ee313264d6)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:01.958568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d3d92-8ff4-4bee-b2c3-72fe2a2ed902 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:02.063032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1110/2222] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:16:01 2022] POST /identity/v3/projects => generated 371 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:16:02.063885 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65714fc-0dd6-4b35-9edc-ea4463c1963f admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:02.126791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:02.145983 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65714fc-0dd6-4b35-9edc-ea4463c1963f admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:02.157499 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d2a13-4cd9-4b3d-8424-926bf4bbd9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:02.158484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d2a13-4cd9-4b3d-8424-926bf4bbd9ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:02.159018 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d2a13-4cd9-4b3d-8424-926bf4bbd9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:02.159314 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d2a13-4cd9-4b3d-8424-926bf4bbd9ee admin admin] PATH_INFO: `/v3/users/b95ae3519ae74d4b92175f14187c01f7` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:02.160684 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d2a13-4cd9-4b3d-8424-926bf4bbd9ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=b95ae3519ae74d4b92175f14187c01f7)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:02.163258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d2a13-4cd9-4b3d-8424-926bf4bbd9ee admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:02.273716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-180d3d92-8ff4-4bee-b2c3-72fe2a2ed902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72b7d968e5dd4d4d8c6ca2ee313264d6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:02.283170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-180d3d92-8ff4-4bee-b2c3-72fe2a2ed902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72b7d968e5dd4d4d8c6ca2ee313264d6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:02.283982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-180d3d92-8ff4-4bee-b2c3-72fe2a2ed902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72b7d968e5dd4d4d8c6ca2ee313264d6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:02.302138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-180d3d92-8ff4-4bee-b2c3-72fe2a2ed902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72b7d968e5dd4d4d8c6ca2ee313264d6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:02.375808 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-180d3d92-8ff4-4bee-b2c3-72fe2a2ed902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72b7d968e5dd4d4d8c6ca2ee313264d6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:02.395655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-d32d2a13-4cd9-4b3d-8424-926bf4bbd9ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b95ae3519ae74d4b92175f14187c01f7'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:02.396308 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-d32d2a13-4cd9-4b3d-8424-926bf4bbd9ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b95ae3519ae74d4b92175f14187c01f7'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:02.396922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-d32d2a13-4cd9-4b3d-8424-926bf4bbd9ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b95ae3519ae74d4b92175f14187c01f7'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:02.401998 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1113/2223] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:16:01 2022] DELETE /identity/v3/users/72b7d968e5dd4d4d8c6ca2ee313264d6 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:02.406775 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d3d92-8ff4-4bee-b2c3-72fe2a2ed902 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:02.407144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:02.407253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-180d3d92-8ff4-4bee-b2c3-72fe2a2ed902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:02.473418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d3d92-8ff4-4bee-b2c3-72fe2a2ed902 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:02.478693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62eaca3-a49e-41f0-9cdf-c64a8edda1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:02.479590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eaca3-a49e-41f0-9cdf-c64a8edda1dc admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:02.479886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eaca3-a49e-41f0-9cdf-c64a8edda1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:02.480168 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eaca3-a49e-41f0-9cdf-c64a8edda1dc admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:02.480908 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62eaca3-a49e-41f0-9cdf-c64a8edda1dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:02.482941 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62eaca3-a49e-41f0-9cdf-c64a8edda1dc admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:02.521623 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-d32d2a13-4cd9-4b3d-8424-926bf4bbd9ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b95ae3519ae74d4b92175f14187c01f7'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:02.541274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-d32d2a13-4cd9-4b3d-8424-926bf4bbd9ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b95ae3519ae74d4b92175f14187c01f7'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:02.560421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1111/2224] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:16:02 2022] DELETE /identity/v3/users/b95ae3519ae74d4b92175f14187c01f7 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:02.560421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892e22f-e999-4d1a-88f0-672ece08caed None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:02.560421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892e22f-e999-4d1a-88f0-672ece08caed None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:02.560421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892e22f-e999-4d1a-88f0-672ece08caed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:02.577862 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-5892e22f-e999-4d1a-88f0-672ece08caed None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:02.579173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-5892e22f-e999-4d1a-88f0-672ece08caed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:02.615780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1114/2225] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:16:02 2022] POST /identity/v3/users => generated 292 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:16:02.615780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cda2a7-ccce-41f1-a3d7-860cdbe6bba0 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:02.615780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cda2a7-ccce-41f1-a3d7-860cdbe6bba0 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:02.615780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cda2a7-ccce-41f1-a3d7-860cdbe6bba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:02.650170 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1112/2226] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:02.651496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892e22f-e999-4d1a-88f0-672ece08caed None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:02.654858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-5892e22f-e999-4d1a-88f0-672ece08caed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:02.663734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-c7cda2a7-ccce-41f1-a3d7-860cdbe6bba0 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:02.666631 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c7cda2a7-ccce-41f1-a3d7-860cdbe6bba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:02.671803 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892e22f-e999-4d1a-88f0-672ece08caed None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:02.678219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810ee6-1210-4fb9-ae8b-d876ec1ba40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:02.679941 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810ee6-1210-4fb9-ae8b-d876ec1ba40b admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:02.680816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810ee6-1210-4fb9-ae8b-d876ec1ba40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:02.681023 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810ee6-1210-4fb9-ae8b-d876ec1ba40b admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:02.681847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef810ee6-1210-4fb9-ae8b-d876ec1ba40b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:02.683642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef810ee6-1210-4fb9-ae8b-d876ec1ba40b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:02.700346 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1113/2227] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:16:02.700346 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810ee6-1210-4fb9-ae8b-d876ec1ba40b admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:02.702472 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ef810ee6-1210-4fb9-ae8b-d876ec1ba40b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:02.737080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1115/2228] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:02.737882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cda2a7-ccce-41f1-a3d7-860cdbe6bba0 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:02.787050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cda2a7-ccce-41f1-a3d7-860cdbe6bba0 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:02.808493 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc58a2b-5fb0-48e9-8f1b-ba588a6b3e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:02.812901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc58a2b-5fb0-48e9-8f1b-ba588a6b3e9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:02.813523 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc58a2b-5fb0-48e9-8f1b-ba588a6b3e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:02.814437 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc58a2b-5fb0-48e9-8f1b-ba588a6b3e9e admin admin] PATH_INFO: `/v3/projects/cdde0a089cf840d3b4fbd17947fb872d/users/089c15ef31064fee9090685f0302b919/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:02.824676 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810ee6-1210-4fb9-ae8b-d876ec1ba40b admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:02.847468 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077c51f-b610-4f9c-9978-afd6c4caa8d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:02.852145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc58a2b-5fb0-48e9-8f1b-ba588a6b3e9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdde0a089cf840d3b4fbd17947fb872d, user_id=089c15ef31064fee9090685f0302b919, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:02.854063 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077c51f-b610-4f9c-9978-afd6c4caa8d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:02.854501 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077c51f-b610-4f9c-9978-afd6c4caa8d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:02.854832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077c51f-b610-4f9c-9978-afd6c4caa8d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:02.858299 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc58a2b-5fb0-48e9-8f1b-ba588a6b3e9e admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:02.932647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077c51f-b610-4f9c-9978-afd6c4caa8d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:02.937666 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077c51f-b610-4f9c-9978-afd6c4caa8d9 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:02.958617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1114/2229] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:16:02.959397 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077c51f-b610-4f9c-9978-afd6c4caa8d9 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:02.986265 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077c51f-b610-4f9c-9978-afd6c4caa8d9 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:02.993486 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e1cef-0fd1-4572-bd9e-b5c81b378247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:02.995008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e1cef-0fd1-4572-bd9e-b5c81b378247 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:02.995525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e1cef-0fd1-4572-bd9e-b5c81b378247 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:02.998102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e1cef-0fd1-4572-bd9e-b5c81b378247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:03.025763 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1116/2230] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:16:02 2022] PUT /identity/v3/projects/cdde0a089cf840d3b4fbd17947fb872d/users/089c15ef31064fee9090685f0302b919/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:03.025969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc58a2b-5fb0-48e9-8f1b-ba588a6b3e9e admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:03.041401 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e1cef-0fd1-4572-bd9e-b5c81b378247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:03.051699 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e1cef-0fd1-4572-bd9e-b5c81b378247 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:03.087315 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1115/2231] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:16:03.136040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660ca37-9b4b-4b42-9f06-ecff45207817 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:03.136866 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660ca37-9b4b-4b42-9f06-ecff45207817 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:03.137246 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660ca37-9b4b-4b42-9f06-ecff45207817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:03.152993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc58a2b-5fb0-48e9-8f1b-ba588a6b3e9e admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:03.160125 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57cfb9-396c-432f-8ac8-44093823f1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:03.160945 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57cfb9-396c-432f-8ac8-44093823f1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:03.161311 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57cfb9-396c-432f-8ac8-44093823f1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:03.161700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57cfb9-396c-432f-8ac8-44093823f1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:03.162088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-7660ca37-9b4b-4b42-9f06-ecff45207817 None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:03.163137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57cfb9-396c-432f-8ac8-44093823f1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:03.164704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7660ca37-9b4b-4b42-9f06-ecff45207817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:03.165080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57cfb9-396c-432f-8ac8-44093823f1e8 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:03.192604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1117/2232] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:16:03.194822 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57cfb9-396c-432f-8ac8-44093823f1e8 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:03.207335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57cfb9-396c-432f-8ac8-44093823f1e8 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:03.213758 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf5ee8-37f4-41a1-9563-c0e18a5fbae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:03.215187 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf5ee8-37f4-41a1-9563-c0e18a5fbae9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:03.215637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf5ee8-37f4-41a1-9563-c0e18a5fbae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:03.216075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf5ee8-37f4-41a1-9563-c0e18a5fbae9 admin admin] PATH_INFO: `/v3/projects/cdde0a089cf840d3b4fbd17947fb872d/users/089c15ef31064fee9090685f0302b919/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:03.219224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bf5ee8-37f4-41a1-9563-c0e18a5fbae9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdde0a089cf840d3b4fbd17947fb872d, user_id=089c15ef31064fee9090685f0302b919, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:03.223118 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bf5ee8-37f4-41a1-9563-c0e18a5fbae9 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:03.258603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1118/2233] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:16:03 2022] PUT /identity/v3/projects/cdde0a089cf840d3b4fbd17947fb872d/users/089c15ef31064fee9090685f0302b919/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:03.328160 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1116/2234] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Jul 18 15:16:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:03.333367 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:03.344291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf5ee8-37f4-41a1-9563-c0e18a5fbae9 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:03.347310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-11bf5ee8-37f4-41a1-9563-c0e18a5fbae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:03.483997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf5ee8-37f4-41a1-9563-c0e18a5fbae9 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:03.499455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070e332-9db9-40d9-9f34-6754eedbbb71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:03.500147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070e332-9db9-40d9-9f34-6754eedbbb71 service swift] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:03.500439 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070e332-9db9-40d9-9f34-6754eedbbb71 service swift] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:03.500725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070e332-9db9-40d9-9f34-6754eedbbb71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:03.573297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:03.671978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070e332-9db9-40d9-9f34-6754eedbbb71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:03.676190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070e332-9db9-40d9-9f34-6754eedbbb71 service swift] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:03.714972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1119/2235] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:16:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 15:16:04.498992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:04.662420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:04.960847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660ca37-9b4b-4b42-9f06-ecff45207817 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:04.964117 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7660ca37-9b4b-4b42-9f06-ecff45207817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:05.003060 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660ca37-9b4b-4b42-9f06-ecff45207817 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:05.381112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:05.381112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:05.381112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:05.381112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3 admin admin] PATH_INFO: `/v3/projects/1ed01e2b97b7457a857452918a166012` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:05.382405 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ed01e2b97b7457a857452918a166012)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:05.382405 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:05.469372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:06.249347 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ed01e2b97b7457a857452918a166012', 'request_id': 'req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:06.256678 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ed01e2b97b7457a857452918a166012', 'request_id': 'req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:06.263746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ed01e2b97b7457a857452918a166012', 'request_id': 'req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:06.503986 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:06.532832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3 admin admin] The token cache is being invalidate because project 1ed01e2b97b7457a857452918a166012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:16:06.537426 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:06.548322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:06.555107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070e332-9db9-40d9-9f34-6754eedbbb71 service swift] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:06.559196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c070e332-9db9-40d9-9f34-6754eedbbb71 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:06.569507 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1117/2236] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:16:04 2022] DELETE /identity/v3/projects/1ed01e2b97b7457a857452918a166012 => generated 0 bytes in 1610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:06.579740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:06.604541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:06.626322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070e332-9db9-40d9-9f34-6754eedbbb71 service swift] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:06.778093 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5d9c4-72c3-4f62-816d-0e88382ec6a3 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:06.784509 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58adfca-5ff2-4829-a6c2-e08f6adb6fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:06.785479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58adfca-5ff2-4829-a6c2-e08f6adb6fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:06.785762 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58adfca-5ff2-4829-a6c2-e08f6adb6fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:06.786038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58adfca-5ff2-4829-a6c2-e08f6adb6fe8 admin admin] PATH_INFO: `/v3/users/ddb83c487e51468281c64892faf091f6` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:06.788648 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58adfca-5ff2-4829-a6c2-e08f6adb6fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddb83c487e51468281c64892faf091f6)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:06.796071 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58adfca-5ff2-4829-a6c2-e08f6adb6fe8 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:06.883665 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbc71e-5aab-488b-b336-4e8be82e218e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:06.884771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbc71e-5aab-488b-b336-4e8be82e218e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:06.885164 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbc71e-5aab-488b-b336-4e8be82e218e admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:06.885554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbc71e-5aab-488b-b336-4e8be82e218e admin admin] PATH_INFO: `/v3/projects/860223baafd746d8a515d3d6082c3e88` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:06.888580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbc71e-5aab-488b-b336-4e8be82e218e admin admin] RBAC: Authorizing `identity:delete_project(project_id=860223baafd746d8a515d3d6082c3e88)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:06.891535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbc71e-5aab-488b-b336-4e8be82e218e admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:07.034417 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-f58adfca-5ff2-4829-a6c2-e08f6adb6fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddb83c487e51468281c64892faf091f6'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:07.035887 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-f58adfca-5ff2-4829-a6c2-e08f6adb6fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddb83c487e51468281c64892faf091f6'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:07.037707 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-f58adfca-5ff2-4829-a6c2-e08f6adb6fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddb83c487e51468281c64892faf091f6'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:07.167542 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-58cbc71e-5aab-488b-b336-4e8be82e218e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '860223baafd746d8a515d3d6082c3e88', 'request_id': 'req-58cbc71e-5aab-488b-b336-4e8be82e218e'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:07.177343 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-f58adfca-5ff2-4829-a6c2-e08f6adb6fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddb83c487e51468281c64892faf091f6'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:07.185732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-58cbc71e-5aab-488b-b336-4e8be82e218e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '860223baafd746d8a515d3d6082c3e88', 'request_id': 'req-58cbc71e-5aab-488b-b336-4e8be82e218e'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:07.207261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-f58adfca-5ff2-4829-a6c2-e08f6adb6fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddb83c487e51468281c64892faf091f6'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:07.236720 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1118/2237] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:16:06 2022] DELETE /identity/v3/users/ddb83c487e51468281c64892faf091f6 => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:07.276297 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-58cbc71e-5aab-488b-b336-4e8be82e218e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '860223baafd746d8a515d3d6082c3e88', 'request_id': 'req-58cbc71e-5aab-488b-b336-4e8be82e218e'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:07.296640 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-58cbc71e-5aab-488b-b336-4e8be82e218e admin admin] The token cache is being invalidate because project 860223baafd746d8a515d3d6082c3e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:16:07.297062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-58cbc71e-5aab-488b-b336-4e8be82e218e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:07.301454 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-58cbc71e-5aab-488b-b336-4e8be82e218e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:07.309175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1120/2238] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:16:06 2022] DELETE /identity/v3/projects/860223baafd746d8a515d3d6082c3e88 => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:16:07.313824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58adfca-5ff2-4829-a6c2-e08f6adb6fe8 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:07.315544 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-f58adfca-5ff2-4829-a6c2-e08f6adb6fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:07.457248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58adfca-5ff2-4829-a6c2-e08f6adb6fe8 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:07.465426 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5ca5a-19b9-44f7-ac59-bd9725ba85c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:07.466364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5ca5a-19b9-44f7-ac59-bd9725ba85c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:07.467861 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5ca5a-19b9-44f7-ac59-bd9725ba85c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:07.467861 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5ca5a-19b9-44f7-ac59-bd9725ba85c6 admin admin] PATH_INFO: `/v3/users/9f3c597f6bf24d7ba5d46b53b71e8219` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:07.478924 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5ca5a-19b9-44f7-ac59-bd9725ba85c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f3c597f6bf24d7ba5d46b53b71e8219)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:07.482336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5ca5a-19b9-44f7-ac59-bd9725ba85c6 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:07.752470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-80e5ca5a-19b9-44f7-ac59-bd9725ba85c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f3c597f6bf24d7ba5d46b53b71e8219'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:07.758862 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-80e5ca5a-19b9-44f7-ac59-bd9725ba85c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f3c597f6bf24d7ba5d46b53b71e8219'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:07.759565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-80e5ca5a-19b9-44f7-ac59-bd9725ba85c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f3c597f6bf24d7ba5d46b53b71e8219'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:07.836700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-80e5ca5a-19b9-44f7-ac59-bd9725ba85c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f3c597f6bf24d7ba5d46b53b71e8219'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:07.867274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:07.874337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-80e5ca5a-19b9-44f7-ac59-bd9725ba85c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f3c597f6bf24d7ba5d46b53b71e8219'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:07.919474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1119/2239] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:16:07 2022] DELETE /identity/v3/users/9f3c597f6bf24d7ba5d46b53b71e8219 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:08.689160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbc71e-5aab-488b-b336-4e8be82e218e admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:08.692303 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-58cbc71e-5aab-488b-b336-4e8be82e218e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:08.927636 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbc71e-5aab-488b-b336-4e8be82e218e admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:08.958115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:09.078575 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a6f70-22f9-4721-8b39-4a287066b539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:09.079736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a6f70-22f9-4721-8b39-4a287066b539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:09.080037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a6f70-22f9-4721-8b39-4a287066b539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:09.080621 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a6f70-22f9-4721-8b39-4a287066b539 admin admin] PATH_INFO: `/v3/projects/84b530ae96604fa4a55f9be915e9d503` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:09.092836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5ca5a-19b9-44f7-ac59-bd9725ba85c6 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:09.094515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-80e5ca5a-19b9-44f7-ac59-bd9725ba85c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:09.110492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a6f70-22f9-4721-8b39-4a287066b539 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84b530ae96604fa4a55f9be915e9d503)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:09.110492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a6f70-22f9-4721-8b39-4a287066b539 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:09.130150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5ca5a-19b9-44f7-ac59-bd9725ba85c6 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:09.152275 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e2384-b7bb-4dbe-8761-8f71529e4ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:09.156199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e2384-b7bb-4dbe-8761-8f71529e4ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:09.156605 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e2384-b7bb-4dbe-8761-8f71529e4ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:09.157038 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e2384-b7bb-4dbe-8761-8f71529e4ea6 admin admin] PATH_INFO: `/v3/projects/ea7e31f83f8540b6ba4670c7f38c38eb` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:09.163305 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e2384-b7bb-4dbe-8761-8f71529e4ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea7e31f83f8540b6ba4670c7f38c38eb)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:09.169069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e2384-b7bb-4dbe-8761-8f71529e4ea6 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:09.356961 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-339a6f70-22f9-4721-8b39-4a287066b539 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84b530ae96604fa4a55f9be915e9d503', 'request_id': 'req-339a6f70-22f9-4721-8b39-4a287066b539'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:09.378732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-339a6f70-22f9-4721-8b39-4a287066b539 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84b530ae96604fa4a55f9be915e9d503', 'request_id': 'req-339a6f70-22f9-4721-8b39-4a287066b539'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:09.467028 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-bb2e2384-b7bb-4dbe-8761-8f71529e4ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea7e31f83f8540b6ba4670c7f38c38eb', 'request_id': 'req-bb2e2384-b7bb-4dbe-8761-8f71529e4ea6'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:09.479881 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-bb2e2384-b7bb-4dbe-8761-8f71529e4ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea7e31f83f8540b6ba4670c7f38c38eb', 'request_id': 'req-bb2e2384-b7bb-4dbe-8761-8f71529e4ea6'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:09.484978 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-bb2e2384-b7bb-4dbe-8761-8f71529e4ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea7e31f83f8540b6ba4670c7f38c38eb', 'request_id': 'req-bb2e2384-b7bb-4dbe-8761-8f71529e4ea6'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:09.573328 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-339a6f70-22f9-4721-8b39-4a287066b539 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84b530ae96604fa4a55f9be915e9d503', 'request_id': 'req-339a6f70-22f9-4721-8b39-4a287066b539'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:09.618381 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-339a6f70-22f9-4721-8b39-4a287066b539 admin admin] The token cache is being invalidate because project 84b530ae96604fa4a55f9be915e9d503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:16:09.618842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-339a6f70-22f9-4721-8b39-4a287066b539 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:09.619466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-339a6f70-22f9-4721-8b39-4a287066b539 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:09.640127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1121/2240] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:16:08 2022] DELETE /identity/v3/projects/84b530ae96604fa4a55f9be915e9d503 => generated 0 bytes in 934 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:09.640127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bb025-a9dc-43e9-a54f-baaf030ce09f None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:09.640127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bb025-a9dc-43e9-a54f-baaf030ce09f None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:09.640127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bb025-a9dc-43e9-a54f-baaf030ce09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:09.679059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-f36bb025-a9dc-43e9-a54f-baaf030ce09f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:09.681207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f36bb025-a9dc-43e9-a54f-baaf030ce09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:09.779058 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1122/2241] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:09.788678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7265c49-8a75-439a-9154-8be5167513fc None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:09.789935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7265c49-8a75-439a-9154-8be5167513fc None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:09.790253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7265c49-8a75-439a-9154-8be5167513fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:09.814750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-d7265c49-8a75-439a-9154-8be5167513fc None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:09.816279 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d7265c49-8a75-439a-9154-8be5167513fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:09.832135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1123/2242] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:16:09.839269 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7265c49-8a75-439a-9154-8be5167513fc None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:09.855409 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7265c49-8a75-439a-9154-8be5167513fc None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:09.861676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19248d7-ca2b-4c6a-808b-98ee3e790831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:09.862767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19248d7-ca2b-4c6a-808b-98ee3e790831 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:09.863247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19248d7-ca2b-4c6a-808b-98ee3e790831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:09.863734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19248d7-ca2b-4c6a-808b-98ee3e790831 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:09.864782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19248d7-ca2b-4c6a-808b-98ee3e790831 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:16:09.865556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19248d7-ca2b-4c6a-808b-98ee3e790831 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:09.868551 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19248d7-ca2b-4c6a-808b-98ee3e790831 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:09.892347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1124/2243] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:16:09.898379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19248d7-ca2b-4c6a-808b-98ee3e790831 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:09.911519 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19248d7-ca2b-4c6a-808b-98ee3e790831 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:09.920149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a7f7a-b430-4982-a9fe-7ce7e28d9846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:09.921957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a7f7a-b430-4982-a9fe-7ce7e28d9846 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:09.921957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a7f7a-b430-4982-a9fe-7ce7e28d9846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:09.921957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a7f7a-b430-4982-a9fe-7ce7e28d9846 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:09.922394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a7f7a-b430-4982-a9fe-7ce7e28d9846 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:09.924553 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a7f7a-b430-4982-a9fe-7ce7e28d9846 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:10.179194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-bb2e2384-b7bb-4dbe-8761-8f71529e4ea6 admin admin] The token cache is being invalidate because project ea7e31f83f8540b6ba4670c7f38c38eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:16:10.179194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-bb2e2384-b7bb-4dbe-8761-8f71529e4ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:10.179194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-bb2e2384-b7bb-4dbe-8761-8f71529e4ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:10.179194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1120/2244] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:16:09 2022] DELETE /identity/v3/projects/ea7e31f83f8540b6ba4670c7f38c38eb => generated 0 bytes in 1081 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:10.183220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a38cb-f9d7-4514-a3ee-68fe48860258 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:10.183516 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a38cb-f9d7-4514-a3ee-68fe48860258 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:10.184140 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a38cb-f9d7-4514-a3ee-68fe48860258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:10.206624 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-564a38cb-f9d7-4514-a3ee-68fe48860258 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:10.208192 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-564a38cb-f9d7-4514-a3ee-68fe48860258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:10.219309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1121/2245] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:10.228594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac737d-44b9-4712-aa75-16c931ee8302 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:10.229015 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac737d-44b9-4712-aa75-16c931ee8302 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:10.235704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac737d-44b9-4712-aa75-16c931ee8302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:10.265861 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-ebac737d-44b9-4712-aa75-16c931ee8302 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:10.267758 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ebac737d-44b9-4712-aa75-16c931ee8302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:10.280284 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1125/2246] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:16:09 2022] POST /identity/v3/projects => generated 373 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:16:10.287009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a7f7a-b430-4982-a9fe-7ce7e28d9846 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:10.290905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-0d6a7f7a-b430-4982-a9fe-7ce7e28d9846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:10.379116 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:10.452718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1122/2247] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:10.461858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac737d-44b9-4712-aa75-16c931ee8302 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:10.471774 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a7f7a-b430-4982-a9fe-7ce7e28d9846 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:10.481690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac737d-44b9-4712-aa75-16c931ee8302 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:10.484507 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665762e3-c937-4df2-a07c-f3fb33995b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:10.485845 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665762e3-c937-4df2-a07c-f3fb33995b99 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:10.486132 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665762e3-c937-4df2-a07c-f3fb33995b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:10.486417 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665762e3-c937-4df2-a07c-f3fb33995b99 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:10.488154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665762e3-c937-4df2-a07c-f3fb33995b99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:10.490219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665762e3-c937-4df2-a07c-f3fb33995b99 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:10.497155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f943cc5-ce77-43aa-a818-23ad102c6216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:10.497895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f943cc5-ce77-43aa-a818-23ad102c6216 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:10.498246 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f943cc5-ce77-43aa-a818-23ad102c6216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:10.498527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f943cc5-ce77-43aa-a818-23ad102c6216 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:10.499382 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f943cc5-ce77-43aa-a818-23ad102c6216 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:16:10.502710 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f943cc5-ce77-43aa-a818-23ad102c6216 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:10.502710 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f943cc5-ce77-43aa-a818-23ad102c6216 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:10.528213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1123/2248] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:16:10.545567 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f943cc5-ce77-43aa-a818-23ad102c6216 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:10.559460 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f943cc5-ce77-43aa-a818-23ad102c6216 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:10.568615 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbef07b-3d61-490d-91fc-8e5dc131a1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:10.570422 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbef07b-3d61-490d-91fc-8e5dc131a1c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:10.571049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbef07b-3d61-490d-91fc-8e5dc131a1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:10.571537 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbef07b-3d61-490d-91fc-8e5dc131a1c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:10.572471 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbef07b-3d61-490d-91fc-8e5dc131a1c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:10.574949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbef07b-3d61-490d-91fc-8e5dc131a1c8 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:10.626040 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1126/2249] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:16:10 2022] POST /identity/v3/users => generated 287 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:16:10.630500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665762e3-c937-4df2-a07c-f3fb33995b99 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:10.660655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665762e3-c937-4df2-a07c-f3fb33995b99 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:10.676916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a950df8-a6dd-4f8c-9683-f37c08423ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:10.680664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a950df8-a6dd-4f8c-9683-f37c08423ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:10.680664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a950df8-a6dd-4f8c-9683-f37c08423ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:10.680664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a950df8-a6dd-4f8c-9683-f37c08423ad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:10.684342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1124/2250] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:16:10 2022] POST /identity/v3/projects => generated 391 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:16:10.685505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a950df8-a6dd-4f8c-9683-f37c08423ad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:10.688474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbef07b-3d61-490d-91fc-8e5dc131a1c8 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:10.689688 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a950df8-a6dd-4f8c-9683-f37c08423ad4 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:10.724607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1127/2251] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:16:10.739426 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a950df8-a6dd-4f8c-9683-f37c08423ad4 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:10.748005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a950df8-a6dd-4f8c-9683-f37c08423ad4 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:10.831561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cafb1-17bb-43f3-8a59-1a30c9aae727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:10.832280 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cafb1-17bb-43f3-8a59-1a30c9aae727 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:10.832534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cafb1-17bb-43f3-8a59-1a30c9aae727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:10.832814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cafb1-17bb-43f3-8a59-1a30c9aae727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:10.833929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07cafb1-17bb-43f3-8a59-1a30c9aae727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:10.836630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07cafb1-17bb-43f3-8a59-1a30c9aae727 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:10.865292 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1128/2252] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:16:10.870571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cafb1-17bb-43f3-8a59-1a30c9aae727 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:10.886088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cafb1-17bb-43f3-8a59-1a30c9aae727 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:10.908376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b73fd-d038-488e-b52c-7de5291ca94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:10.908376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b73fd-d038-488e-b52c-7de5291ca94e admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:10.908376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b73fd-d038-488e-b52c-7de5291ca94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:10.908376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b73fd-d038-488e-b52c-7de5291ca94e admin admin] PATH_INFO: `/v3/projects/b3fddd4445624ec6922eba1ea5ed3f48/users/cef8b57404e24458b6734dfbb138eeba/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:10.908896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbef07b-3d61-490d-91fc-8e5dc131a1c8 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:10.921131 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b73fd-d038-488e-b52c-7de5291ca94e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3fddd4445624ec6922eba1ea5ed3f48, user_id=cef8b57404e24458b6734dfbb138eeba, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:10.921437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9884c5-3cf4-4ea8-a31d-77fd5017d79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:10.922278 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9884c5-3cf4-4ea8-a31d-77fd5017d79c admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:10.924016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b73fd-d038-488e-b52c-7de5291ca94e admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:10.924575 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9884c5-3cf4-4ea8-a31d-77fd5017d79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:10.925266 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9884c5-3cf4-4ea8-a31d-77fd5017d79c admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:10.926122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9884c5-3cf4-4ea8-a31d-77fd5017d79c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:10.929630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:10.934500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9884c5-3cf4-4ea8-a31d-77fd5017d79c admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:11.032106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1125/2253] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:16:10 2022] POST /identity/v3/users => generated 296 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:16:11.036259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9884c5-3cf4-4ea8-a31d-77fd5017d79c admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:11.048635 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1129/2254] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:16:10 2022] PUT /identity/v3/projects/b3fddd4445624ec6922eba1ea5ed3f48/users/cef8b57404e24458b6734dfbb138eeba/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:11.071934 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9884c5-3cf4-4ea8-a31d-77fd5017d79c admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:11.077552 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95f80b-c2d9-4b75-ac2c-93fd364b502d None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:11.079149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95f80b-c2d9-4b75-ac2c-93fd364b502d None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:11.080604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95f80b-c2d9-4b75-ac2c-93fd364b502d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:11.172756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-ad95f80b-c2d9-4b75-ac2c-93fd364b502d None None] MFA Rules not processed for user `cef8b57404e24458b6734dfbb138eeba`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:11.174024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ad95f80b-c2d9-4b75-ac2c-93fd364b502d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:11.268819 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8776c5e-6b6b-4c8b-9690-b9658a27c63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:11.269842 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8776c5e-6b6b-4c8b-9690-b9658a27c63a admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:11.269842 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8776c5e-6b6b-4c8b-9690-b9658a27c63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:11.270121 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8776c5e-6b6b-4c8b-9690-b9658a27c63a admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:11.270994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8776c5e-6b6b-4c8b-9690-b9658a27c63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:11.278459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8776c5e-6b6b-4c8b-9690-b9658a27c63a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:11.303980 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1126/2255] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:16:11.308184 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8776c5e-6b6b-4c8b-9690-b9658a27c63a admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:11.324713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8776c5e-6b6b-4c8b-9690-b9658a27c63a admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:11.340353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac03c8f-5be4-4e60-9455-dba888f3bac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:11.341332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac03c8f-5be4-4e60-9455-dba888f3bac8 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:11.341880 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac03c8f-5be4-4e60-9455-dba888f3bac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:11.342110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac03c8f-5be4-4e60-9455-dba888f3bac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:11.343040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac03c8f-5be4-4e60-9455-dba888f3bac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:11.349115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac03c8f-5be4-4e60-9455-dba888f3bac8 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:11.364275 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1127/2256] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:16:11.368964 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac03c8f-5be4-4e60-9455-dba888f3bac8 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:11.385499 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac03c8f-5be4-4e60-9455-dba888f3bac8 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:11.408928 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac418c-4390-4fd6-89f0-ab3f10b3de36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:11.410506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac418c-4390-4fd6-89f0-ab3f10b3de36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:11.411203 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac418c-4390-4fd6-89f0-ab3f10b3de36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:11.411740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac418c-4390-4fd6-89f0-ab3f10b3de36 admin admin] PATH_INFO: `/v3/projects/c3106abf707d4ae29daf483203ea5296/users/7798db681f2e4611add4f3815a1901b5/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:11.446467 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac418c-4390-4fd6-89f0-ab3f10b3de36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3106abf707d4ae29daf483203ea5296, user_id=7798db681f2e4611add4f3815a1901b5, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:11.449858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac418c-4390-4fd6-89f0-ab3f10b3de36 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:11.455226 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1130/2257] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:11.457441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:11.469967 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95f80b-c2d9-4b75-ac2c-93fd364b502d None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:11.472803 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ad95f80b-c2d9-4b75-ac2c-93fd364b502d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:11.607296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95f80b-c2d9-4b75-ac2c-93fd364b502d None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:11.613318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1128/2258] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:16:11 2022] PUT /identity/v3/projects/c3106abf707d4ae29daf483203ea5296/users/7798db681f2e4611add4f3815a1901b5/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:11.625595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caff8da-72c5-4c9e-9dcd-573509bb234e None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:11.626339 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caff8da-72c5-4c9e-9dcd-573509bb234e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:11.626870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caff8da-72c5-4c9e-9dcd-573509bb234e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:11.723891 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-9caff8da-72c5-4c9e-9dcd-573509bb234e None None] MFA Rules not processed for user `7798db681f2e4611add4f3815a1901b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:11.732768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9caff8da-72c5-4c9e-9dcd-573509bb234e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:11.817558 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cd273-0bc7-4a97-9143-672d74eea845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:11.818425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cd273-0bc7-4a97-9143-672d74eea845 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:11.818852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cd273-0bc7-4a97-9143-672d74eea845 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:11.819799 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cd273-0bc7-4a97-9143-672d74eea845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:11.835531 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:11.931394 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1129/2259] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:16:11.935660 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caff8da-72c5-4c9e-9dcd-573509bb234e None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:12.026024 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cd273-0bc7-4a97-9143-672d74eea845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:12.041006 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cd273-0bc7-4a97-9143-672d74eea845 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:12.090035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1131/2260] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:16:12.104472 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caff8da-72c5-4c9e-9dcd-573509bb234e None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:12.116705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bedb81-5dba-4525-8262-7db56a151e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:12.117415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bedb81-5dba-4525-8262-7db56a151e7c admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:12.117706 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bedb81-5dba-4525-8262-7db56a151e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:12.117992 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bedb81-5dba-4525-8262-7db56a151e7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:12.121082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bedb81-5dba-4525-8262-7db56a151e7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:12.130796 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bedb81-5dba-4525-8262-7db56a151e7c admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:12.346462 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1130/2261] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:16:11 2022] POST /identity/v3/projects => generated 393 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:16:12.353524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cd273-0bc7-4a97-9143-672d74eea845 service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:12.403062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cd273-0bc7-4a97-9143-672d74eea845 service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:12.596672 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:12.622399 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36424c-b954-494a-8cfc-d025a3f0356b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:12.623544 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36424c-b954-494a-8cfc-d025a3f0356b admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:12.624043 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36424c-b954-494a-8cfc-d025a3f0356b admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:12.624549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36424c-b954-494a-8cfc-d025a3f0356b admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:12.625467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36424c-b954-494a-8cfc-d025a3f0356b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:12.628738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36424c-b954-494a-8cfc-d025a3f0356b admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:12.780260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1132/2262] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:16:12 2022] POST /identity/v3/users => generated 303 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:16:12.780484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bedb81-5dba-4525-8262-7db56a151e7c admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:12.793782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bedb81-5dba-4525-8262-7db56a151e7c admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:12.830366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be9d3a-8219-4d39-8c61-8ca0d9234363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:12.831124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be9d3a-8219-4d39-8c61-8ca0d9234363 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:12.833201 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be9d3a-8219-4d39-8c61-8ca0d9234363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:12.833502 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be9d3a-8219-4d39-8c61-8ca0d9234363 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:12.834283 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be9d3a-8219-4d39-8c61-8ca0d9234363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:12.848880 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be9d3a-8219-4d39-8c61-8ca0d9234363 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:12.898094 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1131/2263] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:16:12.898930 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36424c-b954-494a-8cfc-d025a3f0356b admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:12.914933 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36424c-b954-494a-8cfc-d025a3f0356b admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:12.932986 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040ea3f-9c1d-49fc-9ee7-5382456d3d08 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:12.932986 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040ea3f-9c1d-49fc-9ee7-5382456d3d08 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:12.932986 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040ea3f-9c1d-49fc-9ee7-5382456d3d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:12.937890 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ca0fa-ded1-4f59-a58f-82bbc9c82282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:12.938567 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ca0fa-ded1-4f59-a58f-82bbc9c82282 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:12.944673 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ca0fa-ded1-4f59-a58f-82bbc9c82282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:12.948904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ca0fa-ded1-4f59-a58f-82bbc9c82282 admin admin] PATH_INFO: `/v3/projects/7ab9849d4a2d4f0791cb3a40c390943d/users/41a3e66cbeb245dea39c905d595bc021/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:13.000529 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d040ea3f-9c1d-49fc-9ee7-5382456d3d08 None None] MFA Rules not processed for user `648c869550ee437a8e775e997c741e58`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:13.003050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ca0fa-ded1-4f59-a58f-82bbc9c82282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab9849d4a2d4f0791cb3a40c390943d, user_id=41a3e66cbeb245dea39c905d595bc021, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:13.003236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d040ea3f-9c1d-49fc-9ee7-5382456d3d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:13.005468 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ca0fa-ded1-4f59-a58f-82bbc9c82282 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:13.109130 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:13.138334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1132/2264] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:13.143807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1133/2265] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:16:12 2022] PUT /identity/v3/projects/7ab9849d4a2d4f0791cb3a40c390943d/users/41a3e66cbeb245dea39c905d595bc021/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:13.145284 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040ea3f-9c1d-49fc-9ee7-5382456d3d08 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:13.169788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040ea3f-9c1d-49fc-9ee7-5382456d3d08 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:13.407870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e1ef1-1b03-4426-aacd-9647e35452b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:13.409513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e1ef1-1b03-4426-aacd-9647e35452b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:13.409977 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e1ef1-1b03-4426-aacd-9647e35452b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:13.410612 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e1ef1-1b03-4426-aacd-9647e35452b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:13.411658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e1ef1-1b03-4426-aacd-9647e35452b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:13.416865 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e1ef1-1b03-4426-aacd-9647e35452b4 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:13.451911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ca0fa-ded1-4f59-a58f-82bbc9c82282 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:13.452660 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1133/2266] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:16:13.481656 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ca0fa-ded1-4f59-a58f-82bbc9c82282 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:13.488731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bbe88-d7b9-44ff-b131-876a3df81ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:13.490492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bbe88-d7b9-44ff-b131-876a3df81ece admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:13.492074 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bbe88-d7b9-44ff-b131-876a3df81ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:13.492427 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bbe88-d7b9-44ff-b131-876a3df81ece admin admin] PATH_INFO: `/v3/projects/7ab9849d4a2d4f0791cb3a40c390943d/users/41a3e66cbeb245dea39c905d595bc021/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:13.497993 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bbe88-d7b9-44ff-b131-876a3df81ece admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab9849d4a2d4f0791cb3a40c390943d, user_id=41a3e66cbeb245dea39c905d595bc021, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:13.505084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bbe88-d7b9-44ff-b131-876a3df81ece admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:13.568498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1134/2267] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:16:13 2022] PUT /identity/v3/projects/7ab9849d4a2d4f0791cb3a40c390943d/users/41a3e66cbeb245dea39c905d595bc021/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:16:13.571484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82b7cb-7cd0-4bb9-a494-aaf0e4ded3a5 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:13.572081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82b7cb-7cd0-4bb9-a494-aaf0e4ded3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:13.572549 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82b7cb-7cd0-4bb9-a494-aaf0e4ded3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:13.627479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-bb82b7cb-7cd0-4bb9-a494-aaf0e4ded3a5 None None] MFA Rules not processed for user `41a3e66cbeb245dea39c905d595bc021`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:13.630770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-bb82b7cb-7cd0-4bb9-a494-aaf0e4ded3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:13.879474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1134/2268] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:13.892253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bbe88-d7b9-44ff-b131-876a3df81ece admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:14.047987 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bbe88-d7b9-44ff-b131-876a3df81ece admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:14.055737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4541fbe-d433-4754-a3f9-78e5fa3731dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:14.056562 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4541fbe-d433-4754-a3f9-78e5fa3731dd service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:14.057587 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4541fbe-d433-4754-a3f9-78e5fa3731dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:14.058515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4541fbe-d433-4754-a3f9-78e5fa3731dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:14.075064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4541fbe-d433-4754-a3f9-78e5fa3731dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:14.076876 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4541fbe-d433-4754-a3f9-78e5fa3731dd service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:14.135992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1135/2269] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:16:14.229809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82b7cb-7cd0-4bb9-a494-aaf0e4ded3a5 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:14.247976 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82b7cb-7cd0-4bb9-a494-aaf0e4ded3a5 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:14.263491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed9d57-35de-4b4e-8529-466f08325839 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:14.264191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed9d57-35de-4b4e-8529-466f08325839 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:14.264480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed9d57-35de-4b4e-8529-466f08325839 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:14.264761 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed9d57-35de-4b4e-8529-466f08325839 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:14.367555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4541fbe-d433-4754-a3f9-78e5fa3731dd service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:14.369627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c4541fbe-d433-4754-a3f9-78e5fa3731dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:14.419913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed9d57-35de-4b4e-8529-466f08325839 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:14.421331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed9d57-35de-4b4e-8529-466f08325839 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:14.458032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1135/2270] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:16:14.592913 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4541fbe-d433-4754-a3f9-78e5fa3731dd service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:14.608800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02811b1-7ef1-49a2-88cb-e512b598ec99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76cc0ae48b5a49c6a7a1c11a08c389e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:14.611020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02811b1-7ef1-49a2-88cb-e512b598ec99 service placement] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:14.613609 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02811b1-7ef1-49a2-88cb-e512b598ec99 service placement] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:14.614118 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02811b1-7ef1-49a2-88cb-e512b598ec99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:14.620345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c02811b1-7ef1-49a2-88cb-e512b598ec99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:14.652945 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02811b1-7ef1-49a2-88cb-e512b598ec99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:14.657847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02811b1-7ef1-49a2-88cb-e512b598ec99 service placement] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:14.689435 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1136/2271] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:16:20.007467 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed9d57-35de-4b4e-8529-466f08325839 service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:20.097430 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:20.293225 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed9d57-35de-4b4e-8529-466f08325839 service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:20.303213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5521c9-3d03-4451-9403-c276e516afab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:20.306032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5521c9-3d03-4451-9403-c276e516afab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:20.306890 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5521c9-3d03-4451-9403-c276e516afab admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:20.308115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5521c9-3d03-4451-9403-c276e516afab admin admin] PATH_INFO: `/v3/users/cef8b57404e24458b6734dfbb138eeba` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:20.311153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5521c9-3d03-4451-9403-c276e516afab admin admin] RBAC: Authorizing `identity:delete_user(user_id=cef8b57404e24458b6734dfbb138eeba)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:20.317814 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5521c9-3d03-4451-9403-c276e516afab admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:20.596927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-6b5521c9-3d03-4451-9403-c276e516afab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cef8b57404e24458b6734dfbb138eeba'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:20.597638 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-6b5521c9-3d03-4451-9403-c276e516afab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cef8b57404e24458b6734dfbb138eeba'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:20.598468 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-6b5521c9-3d03-4451-9403-c276e516afab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cef8b57404e24458b6734dfbb138eeba'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:20.711008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-6b5521c9-3d03-4451-9403-c276e516afab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cef8b57404e24458b6734dfbb138eeba'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:20.742057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-6b5521c9-3d03-4451-9403-c276e516afab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cef8b57404e24458b6734dfbb138eeba'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:20.784613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1136/2272] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:16:20 2022] DELETE /identity/v3/users/cef8b57404e24458b6734dfbb138eeba => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:20.789200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84adbd-19a5-40b4-afb9-001a8327ffba None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:20.793787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84adbd-19a5-40b4-afb9-001a8327ffba None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:20.794154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84adbd-19a5-40b4-afb9-001a8327ffba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:20.805609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42c1fc-4e56-46aa-a02a-cab8457c8ed3 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:20.812896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42c1fc-4e56-46aa-a02a-cab8457c8ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:20.812896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42c1fc-4e56-46aa-a02a-cab8457c8ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:20.848175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-dc84adbd-19a5-40b4-afb9-001a8327ffba None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:20.851092 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-dc84adbd-19a5-40b4-afb9-001a8327ffba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:20.877427 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-9e42c1fc-4e56-46aa-a02a-cab8457c8ed3 None None] MFA Rules not processed for user `089c15ef31064fee9090685f0302b919`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:20.911817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9e42c1fc-4e56-46aa-a02a-cab8457c8ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:20.958816 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:21.070573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1137/2273] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:21.083064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84adbd-19a5-40b4-afb9-001a8327ffba None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:21.086755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-dc84adbd-19a5-40b4-afb9-001a8327ffba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:21.252881 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1137/2274] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:20 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:21.285806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84adbd-19a5-40b4-afb9-001a8327ffba None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:21.294380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70352d0-92a4-4e4b-a8a2-00d6c630f149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:21.295133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70352d0-92a4-4e4b-a8a2-00d6c630f149 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:21.295419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70352d0-92a4-4e4b-a8a2-00d6c630f149 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:21.295698 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70352d0-92a4-4e4b-a8a2-00d6c630f149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:21.326577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70352d0-92a4-4e4b-a8a2-00d6c630f149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:21.329225 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70352d0-92a4-4e4b-a8a2-00d6c630f149 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:21.359260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1138/2275] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:16:21.796244 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42c1fc-4e56-46aa-a02a-cab8457c8ed3 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:21.805738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9e42c1fc-4e56-46aa-a02a-cab8457c8ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:21.859258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:21.969116 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42c1fc-4e56-46aa-a02a-cab8457c8ed3 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:21.981308 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12762c1c-ab22-4b40-92f4-31098794ed94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:21.984528 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12762c1c-ab22-4b40-92f4-31098794ed94 service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:21.985223 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12762c1c-ab22-4b40-92f4-31098794ed94 service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:21.986070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12762c1c-ab22-4b40-92f4-31098794ed94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:21.991077 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-12762c1c-ab22-4b40-92f4-31098794ed94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:22.092291 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12762c1c-ab22-4b40-92f4-31098794ed94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:22.096263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12762c1c-ab22-4b40-92f4-31098794ed94 service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:22.127653 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1138/2276] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:16:22.427480 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70352d0-92a4-4e4b-a8a2-00d6c630f149 service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:22.430096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c70352d0-92a4-4e4b-a8a2-00d6c630f149 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:22.463576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70352d0-92a4-4e4b-a8a2-00d6c630f149 service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:22.484235 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d2ddf-23ea-4371-95d0-00ccdc7f167c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:22.485036 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d2ddf-23ea-4371-95d0-00ccdc7f167c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:22.488465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d2ddf-23ea-4371-95d0-00ccdc7f167c admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:22.488751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d2ddf-23ea-4371-95d0-00ccdc7f167c admin admin] PATH_INFO: `/v3/projects/b3fddd4445624ec6922eba1ea5ed3f48` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:22.490553 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d2ddf-23ea-4371-95d0-00ccdc7f167c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3fddd4445624ec6922eba1ea5ed3f48)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:22.496288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d2ddf-23ea-4371-95d0-00ccdc7f167c admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:22.732717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-111d2ddf-23ea-4371-95d0-00ccdc7f167c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3fddd4445624ec6922eba1ea5ed3f48', 'request_id': 'req-111d2ddf-23ea-4371-95d0-00ccdc7f167c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:22.753423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-111d2ddf-23ea-4371-95d0-00ccdc7f167c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3fddd4445624ec6922eba1ea5ed3f48', 'request_id': 'req-111d2ddf-23ea-4371-95d0-00ccdc7f167c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:22.853073 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-111d2ddf-23ea-4371-95d0-00ccdc7f167c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3fddd4445624ec6922eba1ea5ed3f48', 'request_id': 'req-111d2ddf-23ea-4371-95d0-00ccdc7f167c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:22.870423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-111d2ddf-23ea-4371-95d0-00ccdc7f167c admin admin] The token cache is being invalidate because project b3fddd4445624ec6922eba1ea5ed3f48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:16:22.870773 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-111d2ddf-23ea-4371-95d0-00ccdc7f167c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:22.884319 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-111d2ddf-23ea-4371-95d0-00ccdc7f167c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:22.894528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1139/2277] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:16:22 2022] DELETE /identity/v3/projects/b3fddd4445624ec6922eba1ea5ed3f48 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:16:22.896184 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0307cf2-d49e-4566-a7f5-0f23aff4fff6 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:22.897785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0307cf2-d49e-4566-a7f5-0f23aff4fff6 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:22.898372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0307cf2-d49e-4566-a7f5-0f23aff4fff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:22.946455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d0307cf2-d49e-4566-a7f5-0f23aff4fff6 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:22.948539 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d0307cf2-d49e-4566-a7f5-0f23aff4fff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:23.154984 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1139/2278] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:23.156221 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192abc6-93f8-4403-9b04-2dea55f34520 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:23.156521 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192abc6-93f8-4403-9b04-2dea55f34520 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:23.156811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192abc6-93f8-4403-9b04-2dea55f34520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:23.210028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-1192abc6-93f8-4403-9b04-2dea55f34520 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:23.210028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1192abc6-93f8-4403-9b04-2dea55f34520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:23.232251 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1140/2279] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:23.236340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0307cf2-d49e-4566-a7f5-0f23aff4fff6 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:23.252458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0307cf2-d49e-4566-a7f5-0f23aff4fff6 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:23.283392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5fb14-31cb-421f-8f34-140d4219e76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:23.284098 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5fb14-31cb-421f-8f34-140d4219e76f admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:23.284390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5fb14-31cb-421f-8f34-140d4219e76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:23.289515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5fb14-31cb-421f-8f34-140d4219e76f admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:23.289515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5fb14-31cb-421f-8f34-140d4219e76f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:16:23.289515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5fb14-31cb-421f-8f34-140d4219e76f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:23.290692 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5fb14-31cb-421f-8f34-140d4219e76f admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:23.329266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192abc6-93f8-4403-9b04-2dea55f34520 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:23.331636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1140/2280] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:16:23.361424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192abc6-93f8-4403-9b04-2dea55f34520 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:23.386299 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c93f56-5a06-4c69-81d2-76edc0b25718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:23.387014 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c93f56-5a06-4c69-81d2-76edc0b25718 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:23.388022 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c93f56-5a06-4c69-81d2-76edc0b25718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:23.388243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c93f56-5a06-4c69-81d2-76edc0b25718 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:23.393463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c93f56-5a06-4c69-81d2-76edc0b25718 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:23.409631 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c93f56-5a06-4c69-81d2-76edc0b25718 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:23.606461 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1141/2281] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:16:23 2022] POST /identity/v3/projects => generated 367 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:16:23.610119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5fb14-31cb-421f-8f34-140d4219e76f admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:23.633300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5fb14-31cb-421f-8f34-140d4219e76f admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:23.638088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c93f56-5a06-4c69-81d2-76edc0b25718 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:23.640341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-72c93f56-5a06-4c69-81d2-76edc0b25718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:23.740436 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:23.756104 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:23.868691 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76075e8-ee0b-491d-9b36-c319d86c50c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:23.868691 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76075e8-ee0b-491d-9b36-c319d86c50c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:23.868691 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76075e8-ee0b-491d-9b36-c319d86c50c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:23.868691 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76075e8-ee0b-491d-9b36-c319d86c50c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:23.869536 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76075e8-ee0b-491d-9b36-c319d86c50c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:23.869536 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76075e8-ee0b-491d-9b36-c319d86c50c0 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:23.893737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c93f56-5a06-4c69-81d2-76edc0b25718 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:23.941269 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065d308-abfb-4ae7-9f26-5e713626ad5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:23.943317 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065d308-abfb-4ae7-9f26-5e713626ad5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:23.943773 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065d308-abfb-4ae7-9f26-5e713626ad5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:23.944894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065d308-abfb-4ae7-9f26-5e713626ad5b admin admin] PATH_INFO: `/v3/users/7798db681f2e4611add4f3815a1901b5` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:23.951134 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065d308-abfb-4ae7-9f26-5e713626ad5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7798db681f2e4611add4f3815a1901b5)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:23.956715 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065d308-abfb-4ae7-9f26-5e713626ad5b admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:24.279026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1141/2282] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:16:23 2022] POST /identity/v3/users => generated 284 bytes in 669 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:16:24.283124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76075e8-ee0b-491d-9b36-c319d86c50c0 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:24.307122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76075e8-ee0b-491d-9b36-c319d86c50c0 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:24.323388 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69923a00-ff97-4f24-89c5-6e66d2e499ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:24.326462 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69923a00-ff97-4f24-89c5-6e66d2e499ac admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:24.326865 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69923a00-ff97-4f24-89c5-6e66d2e499ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:24.327229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69923a00-ff97-4f24-89c5-6e66d2e499ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:24.328084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69923a00-ff97-4f24-89c5-6e66d2e499ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:24.329875 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69923a00-ff97-4f24-89c5-6e66d2e499ac admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:24.372803 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1142/2283] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:16:24.373915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69923a00-ff97-4f24-89c5-6e66d2e499ac admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:24.407514 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69923a00-ff97-4f24-89c5-6e66d2e499ac admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:24.425207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96b510-e17c-4a93-85c8-35ea6ead3397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:24.425975 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96b510-e17c-4a93-85c8-35ea6ead3397 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:24.426269 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96b510-e17c-4a93-85c8-35ea6ead3397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:24.438589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96b510-e17c-4a93-85c8-35ea6ead3397 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:24.446039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b96b510-e17c-4a93-85c8-35ea6ead3397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:24.446039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b96b510-e17c-4a93-85c8-35ea6ead3397 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:24.480191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1143/2284] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:16:24.482700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96b510-e17c-4a93-85c8-35ea6ead3397 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:24.502175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96b510-e17c-4a93-85c8-35ea6ead3397 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:24.516234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c7213-e00c-45ea-a5c5-8c6442272727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:24.519150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c7213-e00c-45ea-a5c5-8c6442272727 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:24.519448 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c7213-e00c-45ea-a5c5-8c6442272727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:24.520909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c7213-e00c-45ea-a5c5-8c6442272727 admin admin] PATH_INFO: `/v3/projects/693dbef8bddd4f49878d8b16b1d675fd/users/576d17874c2b4cebb8faaf7f19aac800/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:24.560499 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c7213-e00c-45ea-a5c5-8c6442272727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=693dbef8bddd4f49878d8b16b1d675fd, user_id=576d17874c2b4cebb8faaf7f19aac800, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:24.564323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c7213-e00c-45ea-a5c5-8c6442272727 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:24.643996 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9065d308-abfb-4ae7-9f26-5e713626ad5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7798db681f2e4611add4f3815a1901b5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:24.666277 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9065d308-abfb-4ae7-9f26-5e713626ad5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7798db681f2e4611add4f3815a1901b5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:24.667443 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9065d308-abfb-4ae7-9f26-5e713626ad5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7798db681f2e4611add4f3815a1901b5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:24.703815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9065d308-abfb-4ae7-9f26-5e713626ad5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7798db681f2e4611add4f3815a1901b5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:25.130447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9065d308-abfb-4ae7-9f26-5e713626ad5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7798db681f2e4611add4f3815a1901b5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:25.156210 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1142/2285] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:16:23 2022] DELETE /identity/v3/users/7798db681f2e4611add4f3815a1901b5 => generated 0 bytes in 1512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:16:25.156210 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85831a2e-74cb-4423-b282-7006e37e0a0c None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:25.156677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85831a2e-74cb-4423-b282-7006e37e0a0c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:25.156677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85831a2e-74cb-4423-b282-7006e37e0a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:25.195506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1144/2286] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:16:24 2022] PUT /identity/v3/projects/693dbef8bddd4f49878d8b16b1d675fd/users/576d17874c2b4cebb8faaf7f19aac800/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:25.201696 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-85831a2e-74cb-4423-b282-7006e37e0a0c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:25.206982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-85831a2e-74cb-4423-b282-7006e37e0a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:25.279532 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a4aa1-8915-461f-9969-113d556357e0 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:25.280890 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a4aa1-8915-461f-9969-113d556357e0 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:25.281276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a4aa1-8915-461f-9969-113d556357e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:25.341220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-fb2a4aa1-8915-461f-9969-113d556357e0 None None] MFA Rules not processed for user `576d17874c2b4cebb8faaf7f19aac800`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:25.341220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-fb2a4aa1-8915-461f-9969-113d556357e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:25.449323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1143/2287] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:16:25.480357 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85831a2e-74cb-4423-b282-7006e37e0a0c None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:25.481996 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-85831a2e-74cb-4423-b282-7006e37e0a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:25.592567 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1145/2288] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:25.626747 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a4aa1-8915-461f-9969-113d556357e0 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:25.628301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-fb2a4aa1-8915-461f-9969-113d556357e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:25.654708 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85831a2e-74cb-4423-b282-7006e37e0a0c None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:25.668731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e68ea0-b537-45ba-9666-9ca16023dec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:25.674356 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e68ea0-b537-45ba-9666-9ca16023dec9 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:25.674356 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e68ea0-b537-45ba-9666-9ca16023dec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:25.674356 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e68ea0-b537-45ba-9666-9ca16023dec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:25.711593 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e68ea0-b537-45ba-9666-9ca16023dec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:25.714676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e68ea0-b537-45ba-9666-9ca16023dec9 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:25.757101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1144/2289] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:16:25.802910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a4aa1-8915-461f-9969-113d556357e0 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:25.816273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e8a91-55e4-408f-87a3-ac16d85ebe69 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8dcb0a0f2c4a2d955544324213739c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:25.817572 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e8a91-55e4-408f-87a3-ac16d85ebe69 service glance] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:25.820786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e8a91-55e4-408f-87a3-ac16d85ebe69 service glance] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:25.820786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e8a91-55e4-408f-87a3-ac16d85ebe69 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:25.849420 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e8a91-55e4-408f-87a3-ac16d85ebe69 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:25.852261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e8a91-55e4-408f-87a3-ac16d85ebe69 service glance] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:25.900448 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1146/2290] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:25 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:16:26.947923 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e68ea0-b537-45ba-9666-9ca16023dec9 service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:26.947923 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-23e68ea0-b537-45ba-9666-9ca16023dec9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:26.947923 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e68ea0-b537-45ba-9666-9ca16023dec9 service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:26.949452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cffc8-03ee-4bdd-a419-33291d920605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:26.949452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cffc8-03ee-4bdd-a419-33291d920605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:26.949452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cffc8-03ee-4bdd-a419-33291d920605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:26.949452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cffc8-03ee-4bdd-a419-33291d920605 admin admin] PATH_INFO: `/v3/projects/c3106abf707d4ae29daf483203ea5296` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:26.949986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cffc8-03ee-4bdd-a419-33291d920605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3106abf707d4ae29daf483203ea5296)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:26.949986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cffc8-03ee-4bdd-a419-33291d920605 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:27.228874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-af6cffc8-03ee-4bdd-a419-33291d920605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3106abf707d4ae29daf483203ea5296', 'request_id': 'req-af6cffc8-03ee-4bdd-a419-33291d920605'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:27.245367 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-af6cffc8-03ee-4bdd-a419-33291d920605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3106abf707d4ae29daf483203ea5296', 'request_id': 'req-af6cffc8-03ee-4bdd-a419-33291d920605'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:27.338584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-af6cffc8-03ee-4bdd-a419-33291d920605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3106abf707d4ae29daf483203ea5296', 'request_id': 'req-af6cffc8-03ee-4bdd-a419-33291d920605'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:27.363532 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-af6cffc8-03ee-4bdd-a419-33291d920605 admin admin] The token cache is being invalidate because project c3106abf707d4ae29daf483203ea5296 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:16:27.367100 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-af6cffc8-03ee-4bdd-a419-33291d920605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:27.368393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-af6cffc8-03ee-4bdd-a419-33291d920605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:27.370830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1145/2291] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:16:26 2022] DELETE /identity/v3/projects/c3106abf707d4ae29daf483203ea5296 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:27.374802 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e8a91-55e4-408f-87a3-ac16d85ebe69 service glance] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:27.398609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e8a91-55e4-408f-87a3-ac16d85ebe69 service glance] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:27.437032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:27.524124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc8f5c-1175-4b5d-ab5a-62717aef19ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:27.525948 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc8f5c-1175-4b5d-ab5a-62717aef19ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:27.526248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc8f5c-1175-4b5d-ab5a-62717aef19ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:27.526844 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc8f5c-1175-4b5d-ab5a-62717aef19ef admin admin] PATH_INFO: `/v3/users/41a3e66cbeb245dea39c905d595bc021` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:27.531371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc8f5c-1175-4b5d-ab5a-62717aef19ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=41a3e66cbeb245dea39c905d595bc021)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:27.533577 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc8f5c-1175-4b5d-ab5a-62717aef19ef admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:27.544604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:27.899118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-20bc8f5c-1175-4b5d-ab5a-62717aef19ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41a3e66cbeb245dea39c905d595bc021'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:27.900391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-20bc8f5c-1175-4b5d-ab5a-62717aef19ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41a3e66cbeb245dea39c905d595bc021'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:27.901008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-20bc8f5c-1175-4b5d-ab5a-62717aef19ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41a3e66cbeb245dea39c905d595bc021'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:27.991481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-20bc8f5c-1175-4b5d-ab5a-62717aef19ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41a3e66cbeb245dea39c905d595bc021'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:28.017342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-20bc8f5c-1175-4b5d-ab5a-62717aef19ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41a3e66cbeb245dea39c905d595bc021'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:28.031692 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1147/2292] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:16:27 2022] DELETE /identity/v3/users/41a3e66cbeb245dea39c905d595bc021 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:16:28.371926 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cffc8-03ee-4bdd-a419-33291d920605 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:28.374232 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-af6cffc8-03ee-4bdd-a419-33291d920605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:28.522334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cffc8-03ee-4bdd-a419-33291d920605 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:28.611137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44161152-ca5a-48bc-81bf-da2b7e8e0e61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:28.611998 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44161152-ca5a-48bc-81bf-da2b7e8e0e61 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:28.612537 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44161152-ca5a-48bc-81bf-da2b7e8e0e61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:28.612920 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44161152-ca5a-48bc-81bf-da2b7e8e0e61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:28.616756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-44161152-ca5a-48bc-81bf-da2b7e8e0e61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:28.777171 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44161152-ca5a-48bc-81bf-da2b7e8e0e61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:28.779694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44161152-ca5a-48bc-81bf-da2b7e8e0e61 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:28.796233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1146/2293] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:16:29.087943 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc8f5c-1175-4b5d-ab5a-62717aef19ef admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:29.093459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-20bc8f5c-1175-4b5d-ab5a-62717aef19ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:29.102947 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55369504-55cd-4449-b1a0-fd2552c18bfb None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:29.103546 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55369504-55cd-4449-b1a0-fd2552c18bfb None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:29.104021 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55369504-55cd-4449-b1a0-fd2552c18bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:29.142153 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-55369504-55cd-4449-b1a0-fd2552c18bfb None None] MFA Rules not processed for user `dcfdd71f9b93426bb6b149d2aeb2088e`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:29.143501 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-55369504-55cd-4449-b1a0-fd2552c18bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:29.235106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1147/2294] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:29.247316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc8f5c-1175-4b5d-ab5a-62717aef19ef admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:29.262913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b955975-7e1c-49c2-8f7f-480c0529056a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:29.263908 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b955975-7e1c-49c2-8f7f-480c0529056a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:29.264236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b955975-7e1c-49c2-8f7f-480c0529056a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:29.264560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b955975-7e1c-49c2-8f7f-480c0529056a admin admin] PATH_INFO: `/v3/projects/7ab9849d4a2d4f0791cb3a40c390943d` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:29.266653 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b955975-7e1c-49c2-8f7f-480c0529056a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ab9849d4a2d4f0791cb3a40c390943d)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:29.269381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b955975-7e1c-49c2-8f7f-480c0529056a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:29.487546 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-2b955975-7e1c-49c2-8f7f-480c0529056a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ab9849d4a2d4f0791cb3a40c390943d', 'request_id': 'req-2b955975-7e1c-49c2-8f7f-480c0529056a'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:29.496407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-2b955975-7e1c-49c2-8f7f-480c0529056a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ab9849d4a2d4f0791cb3a40c390943d', 'request_id': 'req-2b955975-7e1c-49c2-8f7f-480c0529056a'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:29.512411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-2b955975-7e1c-49c2-8f7f-480c0529056a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ab9849d4a2d4f0791cb3a40c390943d', 'request_id': 'req-2b955975-7e1c-49c2-8f7f-480c0529056a'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:29.542352 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2caf6b9-778a-4444-a641-a89625a4ed9b None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:29.543273 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2caf6b9-778a-4444-a641-a89625a4ed9b None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:29.543273 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2caf6b9-778a-4444-a641-a89625a4ed9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:29.568871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-a2caf6b9-778a-4444-a641-a89625a4ed9b None None] MFA Rules not processed for user `faa3eace09844298a4cf0373ddc74de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:29.571102 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a2caf6b9-778a-4444-a641-a89625a4ed9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:29.631906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-2b955975-7e1c-49c2-8f7f-480c0529056a admin admin] The token cache is being invalidate because project 7ab9849d4a2d4f0791cb3a40c390943d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:16:29.632922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-2b955975-7e1c-49c2-8f7f-480c0529056a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:29.634775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-2b955975-7e1c-49c2-8f7f-480c0529056a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:29.641923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1148/2295] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:16:29 2022] DELETE /identity/v3/projects/7ab9849d4a2d4f0791cb3a40c390943d => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:29.644319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4639449-d75b-445f-8286-9362e854f169 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:29.644518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4639449-d75b-445f-8286-9362e854f169 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:29.645139 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4639449-d75b-445f-8286-9362e854f169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:29.670431 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1148/2296] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:16:29.670688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b4639449-d75b-445f-8286-9362e854f169 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:29.671817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b4639449-d75b-445f-8286-9362e854f169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:29.801883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1149/2297] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:29.813688 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7938e-87f7-4e35-a31e-cee826cd5e5d None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:29.814008 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7938e-87f7-4e35-a31e-cee826cd5e5d None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:29.814303 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7938e-87f7-4e35-a31e-cee826cd5e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:29.855340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-dfe7938e-87f7-4e35-a31e-cee826cd5e5d None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:29.857883 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-dfe7938e-87f7-4e35-a31e-cee826cd5e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:29.873964 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1149/2298] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:16:29.877635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4639449-d75b-445f-8286-9362e854f169 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:29.890084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4639449-d75b-445f-8286-9362e854f169 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:29.905588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2e0e3-2e83-486e-88e0-c2c65b087309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:29.906414 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2e0e3-2e83-486e-88e0-c2c65b087309 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:29.906895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2e0e3-2e83-486e-88e0-c2c65b087309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:29.907274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2e0e3-2e83-486e-88e0-c2c65b087309 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:29.911619 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2e0e3-2e83-486e-88e0-c2c65b087309 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:16:29.912036 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2e0e3-2e83-486e-88e0-c2c65b087309 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:29.918183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2e0e3-2e83-486e-88e0-c2c65b087309 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:29.953050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1150/2299] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:16:29.965046 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7938e-87f7-4e35-a31e-cee826cd5e5d None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:29.985669 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7938e-87f7-4e35-a31e-cee826cd5e5d None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:29.995931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e401b5-5461-4db5-87b9-7f776d53048c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:29.997310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e401b5-5461-4db5-87b9-7f776d53048c admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:29.997678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e401b5-5461-4db5-87b9-7f776d53048c admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:29.998038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e401b5-5461-4db5-87b9-7f776d53048c admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:30.004588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e401b5-5461-4db5-87b9-7f776d53048c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:30.016688 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e401b5-5461-4db5-87b9-7f776d53048c admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:30.161573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1150/2300] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:16:29 2022] POST /identity/v3/projects => generated 367 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:16:30.168761 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2e0e3-2e83-486e-88e0-c2c65b087309 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:30.185453 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2e0e3-2e83-486e-88e0-c2c65b087309 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:30.354162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4abfdd-f07d-4521-87a6-84b93ea86d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:30.359817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4abfdd-f07d-4521-87a6-84b93ea86d04 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:30.359817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4abfdd-f07d-4521-87a6-84b93ea86d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:30.360128 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4abfdd-f07d-4521-87a6-84b93ea86d04 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:30.360918 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4abfdd-f07d-4521-87a6-84b93ea86d04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:30.362978 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4abfdd-f07d-4521-87a6-84b93ea86d04 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:30.525999 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1151/2301] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:16:30 2022] POST /identity/v3/users => generated 284 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:16:30.526341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e401b5-5461-4db5-87b9-7f776d53048c admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:30.542983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e401b5-5461-4db5-87b9-7f776d53048c admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:30.550428 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57213293-7d06-49d4-b35f-148da6c87cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:30.551190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57213293-7d06-49d4-b35f-148da6c87cfc admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:30.551524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57213293-7d06-49d4-b35f-148da6c87cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:30.551809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57213293-7d06-49d4-b35f-148da6c87cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:30.552629 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57213293-7d06-49d4-b35f-148da6c87cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:30.555084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57213293-7d06-49d4-b35f-148da6c87cfc admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:30.568501 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1151/2302] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:16:30.575451 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4abfdd-f07d-4521-87a6-84b93ea86d04 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:30.600443 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4abfdd-f07d-4521-87a6-84b93ea86d04 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:30.622264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7611966-1bcf-4570-a8dc-ab2da00996fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:30.628704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7611966-1bcf-4570-a8dc-ab2da00996fc admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:30.628704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7611966-1bcf-4570-a8dc-ab2da00996fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:30.629332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7611966-1bcf-4570-a8dc-ab2da00996fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:30.633531 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7611966-1bcf-4570-a8dc-ab2da00996fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:30.637679 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7611966-1bcf-4570-a8dc-ab2da00996fc admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:30.670749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1152/2303] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:16:30.671084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57213293-7d06-49d4-b35f-148da6c87cfc admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:30.699793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57213293-7d06-49d4-b35f-148da6c87cfc admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:30.711623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007bc15-1c16-4f76-a7b3-b5c36c729f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:30.713925 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007bc15-1c16-4f76-a7b3-b5c36c729f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:30.715014 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007bc15-1c16-4f76-a7b3-b5c36c729f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:30.715982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007bc15-1c16-4f76-a7b3-b5c36c729f89 admin admin] PATH_INFO: `/v3/projects/578272b7b4cf4591a011c95f2fa52410/users/fc857de1198843719260f890268488ab/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:30.761709 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007bc15-1c16-4f76-a7b3-b5c36c729f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=578272b7b4cf4591a011c95f2fa52410, user_id=fc857de1198843719260f890268488ab, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:30.767811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007bc15-1c16-4f76-a7b3-b5c36c729f89 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:30.947414 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:30.959334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1152/2304] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:16:30 2022] PUT /identity/v3/projects/578272b7b4cf4591a011c95f2fa52410/users/fc857de1198843719260f890268488ab/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:16:30.973147 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b3003-5c43-48e1-b13f-ba9ec80101cf None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:30.973559 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b3003-5c43-48e1-b13f-ba9ec80101cf None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:30.973902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b3003-5c43-48e1-b13f-ba9ec80101cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:31.009493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-f09b3003-5c43-48e1-b13f-ba9ec80101cf None None] MFA Rules not processed for user `fc857de1198843719260f890268488ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:31.010973 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-f09b3003-5c43-48e1-b13f-ba9ec80101cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:31.127005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1153/2305] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:31.130971 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007bc15-1c16-4f76-a7b3-b5c36c729f89 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:31.217041 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007bc15-1c16-4f76-a7b3-b5c36c729f89 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:31.222590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aebd75-ec8b-48e4-a0d5-edaeb9eb7818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:31.223581 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aebd75-ec8b-48e4-a0d5-edaeb9eb7818 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:31.223872 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aebd75-ec8b-48e4-a0d5-edaeb9eb7818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:31.224776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aebd75-ec8b-48e4-a0d5-edaeb9eb7818 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:31.224924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aebd75-ec8b-48e4-a0d5-edaeb9eb7818 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:31.227421 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aebd75-ec8b-48e4-a0d5-edaeb9eb7818 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:31.402344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1153/2306] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:16:31 2022] POST /identity/v3/projects => generated 367 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:16:31.406418 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b3003-5c43-48e1-b13f-ba9ec80101cf None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:31.418982 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b3003-5c43-48e1-b13f-ba9ec80101cf None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:31.497864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b09bb-13d3-499f-b04b-7ef8b2d979d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:31.498651 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b09bb-13d3-499f-b04b-7ef8b2d979d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:31.498945 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b09bb-13d3-499f-b04b-7ef8b2d979d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:31.499229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b09bb-13d3-499f-b04b-7ef8b2d979d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:31.499973 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b09bb-13d3-499f-b04b-7ef8b2d979d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:31.502017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b09bb-13d3-499f-b04b-7ef8b2d979d8 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:31.606056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1154/2307] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:16:31 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:16:31.612716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aebd75-ec8b-48e4-a0d5-edaeb9eb7818 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:31.633917 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aebd75-ec8b-48e4-a0d5-edaeb9eb7818 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:31.640948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96350d-794f-4acb-a0a4-f5b3412e21b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:31.641644 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96350d-794f-4acb-a0a4-f5b3412e21b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:31.641935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96350d-794f-4acb-a0a4-f5b3412e21b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:31.642214 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96350d-794f-4acb-a0a4-f5b3412e21b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:31.643039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96350d-794f-4acb-a0a4-f5b3412e21b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:31.644704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96350d-794f-4acb-a0a4-f5b3412e21b0 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:31.657983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1154/2308] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:16:31.664554 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b09bb-13d3-499f-b04b-7ef8b2d979d8 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:31.676667 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b09bb-13d3-499f-b04b-7ef8b2d979d8 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:31.685510 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb44b1-2877-485c-8c26-630347ba2971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:31.686935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb44b1-2877-485c-8c26-630347ba2971 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:31.687457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb44b1-2877-485c-8c26-630347ba2971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:31.687941 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb44b1-2877-485c-8c26-630347ba2971 admin admin] PATH_INFO: `/v3/projects/945881b3253746269ef5afd575a12dd8/users/40dc5ea5e32643f9a78f32478e3450ec/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:31.706464 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb44b1-2877-485c-8c26-630347ba2971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=945881b3253746269ef5afd575a12dd8, user_id=40dc5ea5e32643f9a78f32478e3450ec, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:31.710290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb44b1-2877-485c-8c26-630347ba2971 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:31.797817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1155/2309] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:16:31 2022] PUT /identity/v3/projects/945881b3253746269ef5afd575a12dd8/users/40dc5ea5e32643f9a78f32478e3450ec/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:31.802015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96350d-794f-4acb-a0a4-f5b3412e21b0 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:31.818160 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96350d-794f-4acb-a0a4-f5b3412e21b0 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:31.887674 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425bb24-a971-440b-be77-3497507ce75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:31.888707 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425bb24-a971-440b-be77-3497507ce75e admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:31.889053 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425bb24-a971-440b-be77-3497507ce75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:31.889271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425bb24-a971-440b-be77-3497507ce75e admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:31.890032 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d425bb24-a971-440b-be77-3497507ce75e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:31.892538 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d425bb24-a971-440b-be77-3497507ce75e admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:31.910044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1155/2310] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:16:31.914869 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb44b1-2877-485c-8c26-630347ba2971 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:31.941344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb44b1-2877-485c-8c26-630347ba2971 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:31.963130 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6aa13-7fb0-489e-b98b-9a0c66584998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:31.964081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6aa13-7fb0-489e-b98b-9a0c66584998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:31.964367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6aa13-7fb0-489e-b98b-9a0c66584998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:31.964896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6aa13-7fb0-489e-b98b-9a0c66584998 admin admin] PATH_INFO: `/v3/projects/945881b3253746269ef5afd575a12dd8/users/40dc5ea5e32643f9a78f32478e3450ec/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:31.968770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6aa13-7fb0-489e-b98b-9a0c66584998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=945881b3253746269ef5afd575a12dd8, user_id=40dc5ea5e32643f9a78f32478e3450ec, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:31.972087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6aa13-7fb0-489e-b98b-9a0c66584998 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:31.990349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1156/2311] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:16:31 2022] PUT /identity/v3/projects/945881b3253746269ef5afd575a12dd8/users/40dc5ea5e32643f9a78f32478e3450ec/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:16:32.008742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fab552-e1cd-4bec-b29a-5e0afa954764 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:32.009226 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fab552-e1cd-4bec-b29a-5e0afa954764 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:32.009513 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fab552-e1cd-4bec-b29a-5e0afa954764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:32.043321 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-01fab552-e1cd-4bec-b29a-5e0afa954764 None None] MFA Rules not processed for user `40dc5ea5e32643f9a78f32478e3450ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:32.044598 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-01fab552-e1cd-4bec-b29a-5e0afa954764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:32.195971 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1156/2312] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:32 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:32.217003 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6aa13-7fb0-489e-b98b-9a0c66584998 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:32.219589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c2d6aa13-7fb0-489e-b98b-9a0c66584998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:32.302268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6aa13-7fb0-489e-b98b-9a0c66584998 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:32.309846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a3a55-2433-4959-b081-b0cf3e386841 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:32.311484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a3a55-2433-4959-b081-b0cf3e386841 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:32.311875 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a3a55-2433-4959-b081-b0cf3e386841 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:32.312359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a3a55-2433-4959-b081-b0cf3e386841 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:32.328303 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a3a55-2433-4959-b081-b0cf3e386841 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:32.330600 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a3a55-2433-4959-b081-b0cf3e386841 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:32.347064 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1157/2313] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:32 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:16:32.736116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fab552-e1cd-4bec-b29a-5e0afa954764 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:32.736116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-01fab552-e1cd-4bec-b29a-5e0afa954764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:32.751260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fab552-e1cd-4bec-b29a-5e0afa954764 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:32.757991 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:32.773269 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e768346-8536-4b25-b0f5-e0c20de337f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:32.774590 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e768346-8536-4b25-b0f5-e0c20de337f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:32.775082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e768346-8536-4b25-b0f5-e0c20de337f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:32.775587 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e768346-8536-4b25-b0f5-e0c20de337f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:32.855142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e768346-8536-4b25-b0f5-e0c20de337f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:32.856788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e768346-8536-4b25-b0f5-e0c20de337f0 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:32.868809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1157/2314] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:32 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:16:35.033227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0854c95-6296-474a-a39a-62ae86d0df60 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:35.034375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0854c95-6296-474a-a39a-62ae86d0df60 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:35.034375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0854c95-6296-474a-a39a-62ae86d0df60 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:35.035967 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1158/2315] 10.222.0.83 () {54 vars in 921 bytes} [Mon Jul 18 15:16:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 15:16:35.044970 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793a9e0-2377-43b3-826d-7c2aa7335867 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:35.046126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793a9e0-2377-43b3-826d-7c2aa7335867 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:35.046126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793a9e0-2377-43b3-826d-7c2aa7335867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:35.098005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-2793a9e0-2377-43b3-826d-7c2aa7335867 None None] MFA Rules not processed for user `7e1355d18fb049468651b3675c217d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:35.101655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-2793a9e0-2377-43b3-826d-7c2aa7335867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:35.219704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1158/2316] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Jul 18 15:16:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:16:35.485417 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:37.900570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:39.747193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:42.823143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0854c95-6296-474a-a39a-62ae86d0df60 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:42.865856 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:42.917389 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0854c95-6296-474a-a39a-62ae86d0df60 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:42.923361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe85aee-af75-4755-a7a8-6befc16cf642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:42.924207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe85aee-af75-4755-a7a8-6befc16cf642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:42.924519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe85aee-af75-4755-a7a8-6befc16cf642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:42.926710 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe85aee-af75-4755-a7a8-6befc16cf642 admin admin] PATH_INFO: `/v3/users/fc857de1198843719260f890268488ab` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:42.927557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe85aee-af75-4755-a7a8-6befc16cf642 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc857de1198843719260f890268488ab)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:42.932498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe85aee-af75-4755-a7a8-6befc16cf642 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:43.208813 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-dfe85aee-af75-4755-a7a8-6befc16cf642 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc857de1198843719260f890268488ab'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:43.211803 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-dfe85aee-af75-4755-a7a8-6befc16cf642 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc857de1198843719260f890268488ab'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:43.213730 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-dfe85aee-af75-4755-a7a8-6befc16cf642 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc857de1198843719260f890268488ab'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:43.327573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-dfe85aee-af75-4755-a7a8-6befc16cf642 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc857de1198843719260f890268488ab'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:43.343030 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-dfe85aee-af75-4755-a7a8-6befc16cf642 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc857de1198843719260f890268488ab'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:43.358337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1159/2317] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:16:42 2022] DELETE /identity/v3/users/fc857de1198843719260f890268488ab => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:43.364738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b43fb-0368-4013-8153-f4bf0376092a None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:43.365331 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b43fb-0368-4013-8153-f4bf0376092a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:43.365331 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b43fb-0368-4013-8153-f4bf0376092a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:43.394750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-3f6b43fb-0368-4013-8153-f4bf0376092a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:43.396304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3f6b43fb-0368-4013-8153-f4bf0376092a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:43.492008 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1159/2318] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:43.503403 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe85aee-af75-4755-a7a8-6befc16cf642 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:43.505328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-dfe85aee-af75-4755-a7a8-6befc16cf642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:43.576791 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe85aee-af75-4755-a7a8-6befc16cf642 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:43.581005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3941265-2df0-45e2-87cb-c9591fcb7ec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:43.581806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3941265-2df0-45e2-87cb-c9591fcb7ec2 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:43.582094 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3941265-2df0-45e2-87cb-c9591fcb7ec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:43.582378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3941265-2df0-45e2-87cb-c9591fcb7ec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:43.594593 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3941265-2df0-45e2-87cb-c9591fcb7ec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:43.596110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3941265-2df0-45e2-87cb-c9591fcb7ec2 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:43.611748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1160/2319] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:16:44.268756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b43fb-0368-4013-8153-f4bf0376092a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:44.276649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3f6b43fb-0368-4013-8153-f4bf0376092a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:44.339953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b43fb-0368-4013-8153-f4bf0376092a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:44.351648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b911da7-fd70-4804-962f-099883cce600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:44.352815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b911da7-fd70-4804-962f-099883cce600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:44.353217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b911da7-fd70-4804-962f-099883cce600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:44.353600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b911da7-fd70-4804-962f-099883cce600 admin admin] PATH_INFO: `/v3/projects/578272b7b4cf4591a011c95f2fa52410` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:44.355448 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b911da7-fd70-4804-962f-099883cce600 admin admin] RBAC: Authorizing `identity:delete_project(project_id=578272b7b4cf4591a011c95f2fa52410)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:44.357919 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b911da7-fd70-4804-962f-099883cce600 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:44.524684 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3941265-2df0-45e2-87cb-c9591fcb7ec2 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:44.528043 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d3941265-2df0-45e2-87cb-c9591fcb7ec2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:44.550169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7b911da7-fd70-4804-962f-099883cce600 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '578272b7b4cf4591a011c95f2fa52410', 'request_id': 'req-7b911da7-fd70-4804-962f-099883cce600'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:44.565062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7b911da7-fd70-4804-962f-099883cce600 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '578272b7b4cf4591a011c95f2fa52410', 'request_id': 'req-7b911da7-fd70-4804-962f-099883cce600'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:44.573839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:44.609162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:44.617896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3941265-2df0-45e2-87cb-c9591fcb7ec2 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:44.666385 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7b911da7-fd70-4804-962f-099883cce600 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '578272b7b4cf4591a011c95f2fa52410', 'request_id': 'req-7b911da7-fd70-4804-962f-099883cce600'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:44.687008 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7b911da7-fd70-4804-962f-099883cce600 admin admin] The token cache is being invalidate because project 578272b7b4cf4591a011c95f2fa52410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:16:44.687577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7b911da7-fd70-4804-962f-099883cce600 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:44.688679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7b911da7-fd70-4804-962f-099883cce600 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:44.690441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b234cf4-41da-461d-9ad4-8e99555a0dd3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8dcb0a0f2c4a2d955544324213739c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:44.693357 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b234cf4-41da-461d-9ad4-8e99555a0dd3 service glance] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:44.693357 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b234cf4-41da-461d-9ad4-8e99555a0dd3 service glance] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:44.693593 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1160/2320] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:16:44 2022] DELETE /identity/v3/projects/578272b7b4cf4591a011c95f2fa52410 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:44.693858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b234cf4-41da-461d-9ad4-8e99555a0dd3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:44.698004 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9b234cf4-41da-461d-9ad4-8e99555a0dd3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:44.700300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b911da7-fd70-4804-962f-099883cce600 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:44.702614 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7b911da7-fd70-4804-962f-099883cce600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:44.822598 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b234cf4-41da-461d-9ad4-8e99555a0dd3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:44.823454 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b911da7-fd70-4804-962f-099883cce600 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:44.824548 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b234cf4-41da-461d-9ad4-8e99555a0dd3 service glance] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:44.829359 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8b413-a459-4c12-af75-70079eaba43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:44.830271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8b413-a459-4c12-af75-70079eaba43b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:44.830583 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8b413-a459-4c12-af75-70079eaba43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:44.830866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8b413-a459-4c12-af75-70079eaba43b admin admin] PATH_INFO: `/v3/users/40dc5ea5e32643f9a78f32478e3450ec` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:44.832209 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff8b413-a459-4c12-af75-70079eaba43b admin admin] RBAC: Authorizing `identity:delete_user(user_id=40dc5ea5e32643f9a78f32478e3450ec)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:44.834227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff8b413-a459-4c12-af75-70079eaba43b admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:44.839400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1161/2321] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:16:45.085514 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-cff8b413-a459-4c12-af75-70079eaba43b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40dc5ea5e32643f9a78f32478e3450ec'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:45.099828 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-cff8b413-a459-4c12-af75-70079eaba43b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40dc5ea5e32643f9a78f32478e3450ec'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:45.100776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-cff8b413-a459-4c12-af75-70079eaba43b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40dc5ea5e32643f9a78f32478e3450ec'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:45.124916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-cff8b413-a459-4c12-af75-70079eaba43b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40dc5ea5e32643f9a78f32478e3450ec'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:45.152206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94348875-5668-40f9-805d-4f4b99858777 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:45.152801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94348875-5668-40f9-805d-4f4b99858777 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:45.153403 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94348875-5668-40f9-805d-4f4b99858777 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:45.156279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1162/2322] 10.222.0.83 () {54 vars in 921 bytes} [Mon Jul 18 15:16:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:16:45.169142 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8b796-296d-4c3f-9a62-08ef89e0624e None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:45.170508 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8b796-296d-4c3f-9a62-08ef89e0624e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:45.170508 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8b796-296d-4c3f-9a62-08ef89e0624e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:45.222008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-22e8b796-296d-4c3f-9a62-08ef89e0624e None None] MFA Rules not processed for user `fd86979e0f0d4a4e9921ee5704901f74`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:45.226834 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-22e8b796-296d-4c3f-9a62-08ef89e0624e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:45.253666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-cff8b413-a459-4c12-af75-70079eaba43b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40dc5ea5e32643f9a78f32478e3450ec'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:45.270707 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1161/2323] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:16:44 2022] DELETE /identity/v3/users/40dc5ea5e32643f9a78f32478e3450ec => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:16:45.328258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1163/2324] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Jul 18 15:16:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:45.349519 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8b413-a459-4c12-af75-70079eaba43b admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:45.352030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-cff8b413-a459-4c12-af75-70079eaba43b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:45.457709 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8b413-a459-4c12-af75-70079eaba43b admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:45.467499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c96a29-eecf-48fb-9f60-a4ceb0019f10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:45.468803 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c96a29-eecf-48fb-9f60-a4ceb0019f10 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:45.469321 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c96a29-eecf-48fb-9f60-a4ceb0019f10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:45.470075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c96a29-eecf-48fb-9f60-a4ceb0019f10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:45.486727 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c96a29-eecf-48fb-9f60-a4ceb0019f10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:45.489334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c96a29-eecf-48fb-9f60-a4ceb0019f10 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:45.501477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1162/2325] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:16:45.686652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52c41b-96fa-4bf3-a383-6b558d82484d None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:45.687763 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52c41b-96fa-4bf3-a383-6b558d82484d None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:45.688047 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52c41b-96fa-4bf3-a383-6b558d82484d None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:45.690010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1164/2326] 10.222.0.83 () {54 vars in 921 bytes} [Mon Jul 18 15:16:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:16:45.698660 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b50280-a4dc-48bc-8926-4b48de8e05a9 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:45.699651 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b50280-a4dc-48bc-8926-4b48de8e05a9 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:45.699651 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b50280-a4dc-48bc-8926-4b48de8e05a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:45.732430 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-e8b50280-a4dc-48bc-8926-4b48de8e05a9 None None] MFA Rules not processed for user `7e1355d18fb049468651b3675c217d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:45.737702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e8b50280-a4dc-48bc-8926-4b48de8e05a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:45.849059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1163/2327] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Jul 18 15:16:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:45.866904 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52c41b-96fa-4bf3-a383-6b558d82484d None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:45.880251 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52c41b-96fa-4bf3-a383-6b558d82484d None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:45.885463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6f824-45fc-487a-8cfe-88506757f303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:45.886285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6f824-45fc-487a-8cfe-88506757f303 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:45.886606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6f824-45fc-487a-8cfe-88506757f303 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:45.886896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6f824-45fc-487a-8cfe-88506757f303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:45.899625 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6f824-45fc-487a-8cfe-88506757f303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:45.902885 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6f824-45fc-487a-8cfe-88506757f303 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:45.914590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1165/2328] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:16:45.932239 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b50280-a4dc-48bc-8926-4b48de8e05a9 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:45.935147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e8b50280-a4dc-48bc-8926-4b48de8e05a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:46.048411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b50280-a4dc-48bc-8926-4b48de8e05a9 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:46.056809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793b8a7-8257-42cf-94b4-d52e92d5291b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:46.058416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793b8a7-8257-42cf-94b4-d52e92d5291b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:46.058973 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793b8a7-8257-42cf-94b4-d52e92d5291b admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:46.063980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793b8a7-8257-42cf-94b4-d52e92d5291b admin admin] PATH_INFO: `/v3/projects/945881b3253746269ef5afd575a12dd8` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:46.063980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793b8a7-8257-42cf-94b4-d52e92d5291b admin admin] RBAC: Authorizing `identity:delete_project(project_id=945881b3253746269ef5afd575a12dd8)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:46.066003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793b8a7-8257-42cf-94b4-d52e92d5291b admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:46.233597 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2793b8a7-8257-42cf-94b4-d52e92d5291b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '945881b3253746269ef5afd575a12dd8', 'request_id': 'req-2793b8a7-8257-42cf-94b4-d52e92d5291b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:46.241619 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2793b8a7-8257-42cf-94b4-d52e92d5291b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '945881b3253746269ef5afd575a12dd8', 'request_id': 'req-2793b8a7-8257-42cf-94b4-d52e92d5291b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:46.390981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2793b8a7-8257-42cf-94b4-d52e92d5291b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '945881b3253746269ef5afd575a12dd8', 'request_id': 'req-2793b8a7-8257-42cf-94b4-d52e92d5291b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:46.416973 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2793b8a7-8257-42cf-94b4-d52e92d5291b admin admin] The token cache is being invalidate because project 945881b3253746269ef5afd575a12dd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:16:46.417853 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2793b8a7-8257-42cf-94b4-d52e92d5291b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:46.422376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2793b8a7-8257-42cf-94b4-d52e92d5291b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:16:46.431894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1164/2329] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:16:45 2022] DELETE /identity/v3/projects/945881b3253746269ef5afd575a12dd8 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:46.435418 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990d2d9-8150-473f-8635-fdb5eb540292 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:46.436261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990d2d9-8150-473f-8635-fdb5eb540292 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:46.436424 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990d2d9-8150-473f-8635-fdb5eb540292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:46.466170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:46.487087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-0990d2d9-8150-473f-8635-fdb5eb540292 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:46.489594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0990d2d9-8150-473f-8635-fdb5eb540292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:46.566409 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:46.625826 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1166/2330] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:46.644777 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cdd2d-c3fd-42a5-8b23-5f7b09e08a5f None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:46.644777 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cdd2d-c3fd-42a5-8b23-5f7b09e08a5f None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:46.644777 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cdd2d-c3fd-42a5-8b23-5f7b09e08a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:46.673138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-d16cdd2d-c3fd-42a5-8b23-5f7b09e08a5f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:46.674583 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d16cdd2d-c3fd-42a5-8b23-5f7b09e08a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:46.684362 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1165/2331] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:46.689471 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990d2d9-8150-473f-8635-fdb5eb540292 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:46.701421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990d2d9-8150-473f-8635-fdb5eb540292 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:46.706681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe98924-af9b-4c5d-a7b5-23fa21884d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:46.707370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe98924-af9b-4c5d-a7b5-23fa21884d56 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:46.707656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe98924-af9b-4c5d-a7b5-23fa21884d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:46.707936 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe98924-af9b-4c5d-a7b5-23fa21884d56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:46.708715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe98924-af9b-4c5d-a7b5-23fa21884d56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:16:46.709165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe98924-af9b-4c5d-a7b5-23fa21884d56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:46.711578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe98924-af9b-4c5d-a7b5-23fa21884d56 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:46.730484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1167/2332] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:16:46.744445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cdd2d-c3fd-42a5-8b23-5f7b09e08a5f None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:46.763460 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cdd2d-c3fd-42a5-8b23-5f7b09e08a5f None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:46.771872 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c4b33-a7e0-44a8-bb72-7790ee4d332a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:46.774232 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c4b33-a7e0-44a8-bb72-7790ee4d332a admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:46.774386 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c4b33-a7e0-44a8-bb72-7790ee4d332a admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:46.774809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c4b33-a7e0-44a8-bb72-7790ee4d332a admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:46.776051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c4b33-a7e0-44a8-bb72-7790ee4d332a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:46.779720 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c4b33-a7e0-44a8-bb72-7790ee4d332a admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:46.907826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1166/2333] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:16:46 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:16:46.913355 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe98924-af9b-4c5d-a7b5-23fa21884d56 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:46.935018 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe98924-af9b-4c5d-a7b5-23fa21884d56 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:47.034879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8ed9c-3f04-4ff8-ac42-b078660d130e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:47.035474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8ed9c-3f04-4ff8-ac42-b078660d130e admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:47.035767 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8ed9c-3f04-4ff8-ac42-b078660d130e admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:47.036051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8ed9c-3f04-4ff8-ac42-b078660d130e admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:47.036767 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8ed9c-3f04-4ff8-ac42-b078660d130e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:47.039966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8ed9c-3f04-4ff8-ac42-b078660d130e admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:47.163953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1168/2334] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:16:46 2022] POST /identity/v3/users => generated 292 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:16:47.169811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c4b33-a7e0-44a8-bb72-7790ee4d332a admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:47.184660 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c4b33-a7e0-44a8-bb72-7790ee4d332a admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:47.190694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465489b-5b12-4e52-ad0a-e97247aa2f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:47.191477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465489b-5b12-4e52-ad0a-e97247aa2f2e admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:47.191827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465489b-5b12-4e52-ad0a-e97247aa2f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:47.192224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465489b-5b12-4e52-ad0a-e97247aa2f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:47.193043 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465489b-5b12-4e52-ad0a-e97247aa2f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:47.196122 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465489b-5b12-4e52-ad0a-e97247aa2f2e admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:47.210521 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1167/2335] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:16:47.215424 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8ed9c-3f04-4ff8-ac42-b078660d130e admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:47.228836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8ed9c-3f04-4ff8-ac42-b078660d130e admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:47.237809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ec480-8e4f-4de1-8459-f5303e81b763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:47.239300 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ec480-8e4f-4de1-8459-f5303e81b763 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:47.239817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ec480-8e4f-4de1-8459-f5303e81b763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:47.240292 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ec480-8e4f-4de1-8459-f5303e81b763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:47.242286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ec480-8e4f-4de1-8459-f5303e81b763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:47.245785 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ec480-8e4f-4de1-8459-f5303e81b763 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:47.260750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1169/2336] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:16:47.265794 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465489b-5b12-4e52-ad0a-e97247aa2f2e admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:47.291544 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465489b-5b12-4e52-ad0a-e97247aa2f2e admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:47.298944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddafcf3-104f-4cbd-bb5e-737738777935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:47.300422 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddafcf3-104f-4cbd-bb5e-737738777935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:47.300926 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddafcf3-104f-4cbd-bb5e-737738777935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:47.301409 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddafcf3-104f-4cbd-bb5e-737738777935 admin admin] PATH_INFO: `/v3/projects/f5210ed64a7e4ac0927370738a21e149/users/e10baeb01cf74431bbbaeab2a8c118a9/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:47.344907 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddafcf3-104f-4cbd-bb5e-737738777935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5210ed64a7e4ac0927370738a21e149, user_id=e10baeb01cf74431bbbaeab2a8c118a9, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:47.351470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddafcf3-104f-4cbd-bb5e-737738777935 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:47.445948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1168/2337] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:16:47 2022] PUT /identity/v3/projects/f5210ed64a7e4ac0927370738a21e149/users/e10baeb01cf74431bbbaeab2a8c118a9/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:16:47.458261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28a2d4-fa8d-4df9-9662-59fd7748c3be None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:47.458584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28a2d4-fa8d-4df9-9662-59fd7748c3be None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:47.459027 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28a2d4-fa8d-4df9-9662-59fd7748c3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:47.508470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-6e28a2d4-fa8d-4df9-9662-59fd7748c3be None None] MFA Rules not processed for user `e10baeb01cf74431bbbaeab2a8c118a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:47.510783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6e28a2d4-fa8d-4df9-9662-59fd7748c3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:47.591717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:47.629870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1170/2338] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:16:47.636324 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddafcf3-104f-4cbd-bb5e-737738777935 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:47.728521 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddafcf3-104f-4cbd-bb5e-737738777935 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:47.737751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbcaf8-037e-4fd1-af37-636e836ff74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:47.739500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbcaf8-037e-4fd1-af37-636e836ff74d admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:47.740358 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbcaf8-037e-4fd1-af37-636e836ff74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:47.740883 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbcaf8-037e-4fd1-af37-636e836ff74d admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:47.743573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbcaf8-037e-4fd1-af37-636e836ff74d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:47.746151 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbcaf8-037e-4fd1-af37-636e836ff74d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:47.869708 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1169/2339] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:16:47 2022] POST /identity/v3/projects => generated 383 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:16:47.875705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28a2d4-fa8d-4df9-9662-59fd7748c3be None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:47.889634 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28a2d4-fa8d-4df9-9662-59fd7748c3be None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:47.950594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:47.999295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee0103-730e-488f-9378-6727d0736765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:48.000080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee0103-730e-488f-9378-6727d0736765 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:48.000296 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee0103-730e-488f-9378-6727d0736765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:48.000578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee0103-730e-488f-9378-6727d0736765 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:48.001331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee0103-730e-488f-9378-6727d0736765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:48.003377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee0103-730e-488f-9378-6727d0736765 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:48.093363 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1171/2340] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:16:47 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:16:48.099196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbcaf8-037e-4fd1-af37-636e836ff74d admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:48.116718 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbcaf8-037e-4fd1-af37-636e836ff74d admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:48.123138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9a3a4-b802-4556-b714-842272e10ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:48.124041 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9a3a4-b802-4556-b714-842272e10ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:48.124452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9a3a4-b802-4556-b714-842272e10ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:48.124911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9a3a4-b802-4556-b714-842272e10ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:48.125997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9a3a4-b802-4556-b714-842272e10ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:48.128121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9a3a4-b802-4556-b714-842272e10ed6 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:48.142679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1170/2341] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:16:48.148236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee0103-730e-488f-9378-6727d0736765 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:48.166357 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee0103-730e-488f-9378-6727d0736765 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:48.172853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80caa094-a45c-4909-87fe-90708263acfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:48.173675 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caa094-a45c-4909-87fe-90708263acfb admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:48.174013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caa094-a45c-4909-87fe-90708263acfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:48.174427 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caa094-a45c-4909-87fe-90708263acfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:48.175340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80caa094-a45c-4909-87fe-90708263acfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:48.177615 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80caa094-a45c-4909-87fe-90708263acfb admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:48.202722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1172/2342] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:16:48.215476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9a3a4-b802-4556-b714-842272e10ed6 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:48.228958 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9a3a4-b802-4556-b714-842272e10ed6 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:48.234113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6a622-82af-4803-8d60-f879896e3002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:48.235351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6a622-82af-4803-8d60-f879896e3002 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:48.235911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6a622-82af-4803-8d60-f879896e3002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:48.236199 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6a622-82af-4803-8d60-f879896e3002 admin admin] PATH_INFO: `/v3/projects/4af97574abb245cba7280db88ac2e249/users/0c345c9595f043d490a1db17ab08214c/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:48.255710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6a622-82af-4803-8d60-f879896e3002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af97574abb245cba7280db88ac2e249, user_id=0c345c9595f043d490a1db17ab08214c, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:48.259399 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6a622-82af-4803-8d60-f879896e3002 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:48.366696 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1171/2343] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:16:48 2022] PUT /identity/v3/projects/4af97574abb245cba7280db88ac2e249/users/0c345c9595f043d490a1db17ab08214c/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:48.385827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa76d74-9df2-4bc0-8362-303921236135 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:48.386148 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa76d74-9df2-4bc0-8362-303921236135 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:48.386416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa76d74-9df2-4bc0-8362-303921236135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:48.427232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-dfa76d74-9df2-4bc0-8362-303921236135 None None] MFA Rules not processed for user `0c345c9595f043d490a1db17ab08214c`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:48.428546 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-dfa76d74-9df2-4bc0-8362-303921236135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:48.529942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1173/2344] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:48.536274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6a622-82af-4803-8d60-f879896e3002 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:48.641578 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6a622-82af-4803-8d60-f879896e3002 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:48.650405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dd237-eb64-4ab0-95fe-8b3393b3948f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:48.651539 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dd237-eb64-4ab0-95fe-8b3393b3948f admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:48.651872 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dd237-eb64-4ab0-95fe-8b3393b3948f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:48.652379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dd237-eb64-4ab0-95fe-8b3393b3948f admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:48.653742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dd237-eb64-4ab0-95fe-8b3393b3948f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:48.657736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dd237-eb64-4ab0-95fe-8b3393b3948f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:48.772193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:48.792011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa76d74-9df2-4bc0-8362-303921236135 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:48.793278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1172/2345] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:16:48 2022] POST /identity/v3/projects => generated 383 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:16:48.813179 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa76d74-9df2-4bc0-8362-303921236135 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:48.935318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e269c3-2d54-4a1a-88e1-74dc03436a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:48.936632 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e269c3-2d54-4a1a-88e1-74dc03436a2e admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:48.937100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e269c3-2d54-4a1a-88e1-74dc03436a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:48.937555 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e269c3-2d54-4a1a-88e1-74dc03436a2e admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:48.938843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e269c3-2d54-4a1a-88e1-74dc03436a2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:48.942374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e269c3-2d54-4a1a-88e1-74dc03436a2e admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:49.070270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1174/2346] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:16:48 2022] POST /identity/v3/users => generated 298 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:16:49.076149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dd237-eb64-4ab0-95fe-8b3393b3948f admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:49.090683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dd237-eb64-4ab0-95fe-8b3393b3948f admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:49.108308 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f3bdf-7852-4269-8a8e-83bd2f5f33d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:49.111158 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f3bdf-7852-4269-8a8e-83bd2f5f33d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:49.111795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f3bdf-7852-4269-8a8e-83bd2f5f33d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:49.112288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f3bdf-7852-4269-8a8e-83bd2f5f33d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:49.113425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f3bdf-7852-4269-8a8e-83bd2f5f33d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:49.115728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f3bdf-7852-4269-8a8e-83bd2f5f33d7 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:49.145564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1173/2347] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:16:49.151917 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e269c3-2d54-4a1a-88e1-74dc03436a2e admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:49.179847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e269c3-2d54-4a1a-88e1-74dc03436a2e admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:49.186777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81473779-5505-4f4b-88db-637ae8e5e197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:49.187770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81473779-5505-4f4b-88db-637ae8e5e197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:49.188064 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81473779-5505-4f4b-88db-637ae8e5e197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:49.188720 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81473779-5505-4f4b-88db-637ae8e5e197 admin admin] PATH_INFO: `/v3/projects/afaa3634d9aa49d984005cfdf964da95/users/95ff51a7c9424982b5d63484b9918f61/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:49.217026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81473779-5505-4f4b-88db-637ae8e5e197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afaa3634d9aa49d984005cfdf964da95, user_id=95ff51a7c9424982b5d63484b9918f61, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:49.223398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81473779-5505-4f4b-88db-637ae8e5e197 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:49.338006 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f3bdf-7852-4269-8a8e-83bd2f5f33d7 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:49.342707 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1175/2348] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:16:49 2022] PUT /identity/v3/projects/afaa3634d9aa49d984005cfdf964da95/users/95ff51a7c9424982b5d63484b9918f61/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:16:49.359704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f3bdf-7852-4269-8a8e-83bd2f5f33d7 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:49.441710 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ffe5e-deaf-4cf0-929a-e8eb851ceeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:49.442476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ffe5e-deaf-4cf0-929a-e8eb851ceeb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:49.442981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ffe5e-deaf-4cf0-929a-e8eb851ceeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:49.443270 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ffe5e-deaf-4cf0-929a-e8eb851ceeb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:49.444129 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ffe5e-deaf-4cf0-929a-e8eb851ceeb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:49.446003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ffe5e-deaf-4cf0-929a-e8eb851ceeb9 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:49.458500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1174/2349] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:16:49.465838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81473779-5505-4f4b-88db-637ae8e5e197 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:49.488204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81473779-5505-4f4b-88db-637ae8e5e197 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:49.497415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d562b-21f0-4759-b471-862632f3a46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:49.498472 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d562b-21f0-4759-b471-862632f3a46c admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:49.498880 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d562b-21f0-4759-b471-862632f3a46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:49.499242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d562b-21f0-4759-b471-862632f3a46c admin admin] PATH_INFO: `/v3/projects/afaa3634d9aa49d984005cfdf964da95/users/95ff51a7c9424982b5d63484b9918f61/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:49.501831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d562b-21f0-4759-b471-862632f3a46c admin admin] RBAC: Authorizing `identity:create_grant(project_id=afaa3634d9aa49d984005cfdf964da95, user_id=95ff51a7c9424982b5d63484b9918f61, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:49.505109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d562b-21f0-4759-b471-862632f3a46c admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:49.511094 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:49.522686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1176/2350] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:16:49 2022] PUT /identity/v3/projects/afaa3634d9aa49d984005cfdf964da95/users/95ff51a7c9424982b5d63484b9918f61/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:16:49.598402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781882f2-1ef9-4a55-bbed-a72e9c987102 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:49.599099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781882f2-1ef9-4a55-bbed-a72e9c987102 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:49.599624 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781882f2-1ef9-4a55-bbed-a72e9c987102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:49.632890 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-781882f2-1ef9-4a55-bbed-a72e9c987102 None None] MFA Rules not processed for user `95ff51a7c9424982b5d63484b9918f61`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:49.634819 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-781882f2-1ef9-4a55-bbed-a72e9c987102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:49.738603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1175/2351] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:16:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:16:49.754125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d562b-21f0-4759-b471-862632f3a46c admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:49.757108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-af9d562b-21f0-4759-b471-862632f3a46c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:49.827729 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d562b-21f0-4759-b471-862632f3a46c admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:49.832014 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc7dcc-dd17-44fd-a465-7846256f5d1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:49.833073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc7dcc-dd17-44fd-a465-7846256f5d1f service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:49.833427 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc7dcc-dd17-44fd-a465-7846256f5d1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:49.833705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc7dcc-dd17-44fd-a465-7846256f5d1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:49.913298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc7dcc-dd17-44fd-a465-7846256f5d1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:49.915174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc7dcc-dd17-44fd-a465-7846256f5d1f service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:49.932377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1177/2352] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:16:50.676924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2544982-e937-4ed9-978d-becb0f56d4af None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:50.677812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2544982-e937-4ed9-978d-becb0f56d4af None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:50.677812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2544982-e937-4ed9-978d-becb0f56d4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:50.704120 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-a2544982-e937-4ed9-978d-becb0f56d4af None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:16:50.704120 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a2544982-e937-4ed9-978d-becb0f56d4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:50.861066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1176/2353] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Jul 18 15:16:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:16:50.896807 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc7dcc-dd17-44fd-a465-7846256f5d1f service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:50.899312 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-dddc7dcc-dd17-44fd-a465-7846256f5d1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:51.006052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc7dcc-dd17-44fd-a465-7846256f5d1f service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:51.012352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da1cb6-fb93-4000-a1fa-1fa2d0587a5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:51.013199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da1cb6-fb93-4000-a1fa-1fa2d0587a5a service swift] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:51.013543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da1cb6-fb93-4000-a1fa-1fa2d0587a5a service swift] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:51.013877 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da1cb6-fb93-4000-a1fa-1fa2d0587a5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:51.026376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da1cb6-fb93-4000-a1fa-1fa2d0587a5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:51.031557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da1cb6-fb93-4000-a1fa-1fa2d0587a5a service swift] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:51.045360 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1178/2354] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:16:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 15:16:52.469396 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2544982-e937-4ed9-978d-becb0f56d4af None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:52.472044 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a2544982-e937-4ed9-978d-becb0f56d4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:16:52.496234 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2544982-e937-4ed9-978d-becb0f56d4af None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:52.502919 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb061943-f96c-4897-ad58-4584e8f5e870 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:52.504267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb061943-f96c-4897-ad58-4584e8f5e870 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:52.504885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb061943-f96c-4897-ad58-4584e8f5e870 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:52.505493 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb061943-f96c-4897-ad58-4584e8f5e870 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:52.531341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:16:52.620724 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb061943-f96c-4897-ad58-4584e8f5e870 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:52.623947 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb061943-f96c-4897-ad58-4584e8f5e870 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:52.644375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1177/2355] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:16:55.615500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da1cb6-fb93-4000-a1fa-1fa2d0587a5a service swift] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:16:55.654274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da1cb6-fb93-4000-a1fa-1fa2d0587a5a service swift] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:16:55.665761 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c13664-058e-4dd4-bcd8-c5b96081342e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:16:55.667642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c13664-058e-4dd4-bcd8-c5b96081342e service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:16:55.668175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c13664-058e-4dd4-bcd8-c5b96081342e service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:16:55.668812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c13664-058e-4dd4-bcd8-c5b96081342e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:16:55.689612 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c13664-058e-4dd4-bcd8-c5b96081342e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:16:55.692731 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c13664-058e-4dd4-bcd8-c5b96081342e service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:16:55.730391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1179/2356] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:16:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:17:02.635596 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=21 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:17:03.561669 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb061943-f96c-4897-ad58-4584e8f5e870 service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:03.664779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb061943-f96c-4897-ad58-4584e8f5e870 service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:03.673140 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bef8e-63b6-431e-95d6-ddf011bacabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:03.674781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bef8e-63b6-431e-95d6-ddf011bacabd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:03.675322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bef8e-63b6-431e-95d6-ddf011bacabd admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:03.675787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bef8e-63b6-431e-95d6-ddf011bacabd admin admin] PATH_INFO: `/v3/users/e10baeb01cf74431bbbaeab2a8c118a9` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:03.677885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bef8e-63b6-431e-95d6-ddf011bacabd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e10baeb01cf74431bbbaeab2a8c118a9)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:03.682353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bef8e-63b6-431e-95d6-ddf011bacabd admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:03.941692 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-559bef8e-63b6-431e-95d6-ddf011bacabd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e10baeb01cf74431bbbaeab2a8c118a9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:03.952287 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-559bef8e-63b6-431e-95d6-ddf011bacabd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e10baeb01cf74431bbbaeab2a8c118a9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:03.953079 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-559bef8e-63b6-431e-95d6-ddf011bacabd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e10baeb01cf74431bbbaeab2a8c118a9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:03.975099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-559bef8e-63b6-431e-95d6-ddf011bacabd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e10baeb01cf74431bbbaeab2a8c118a9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:04.072638 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-559bef8e-63b6-431e-95d6-ddf011bacabd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e10baeb01cf74431bbbaeab2a8c118a9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:04.083648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1178/2357] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:17:03 2022] DELETE /identity/v3/users/e10baeb01cf74431bbbaeab2a8c118a9 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:17:04.091133 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87a468-d01b-4598-87bf-3bdde4974dd3 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:04.091826 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87a468-d01b-4598-87bf-3bdde4974dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:04.092082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87a468-d01b-4598-87bf-3bdde4974dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:04.115303 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-4a87a468-d01b-4598-87bf-3bdde4974dd3 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:17:04.115303 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-4a87a468-d01b-4598-87bf-3bdde4974dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:04.136864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:17:04.183022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1180/2358] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:17:04.195776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bef8e-63b6-431e-95d6-ddf011bacabd admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:04.197634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-559bef8e-63b6-431e-95d6-ddf011bacabd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:04.267865 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bef8e-63b6-431e-95d6-ddf011bacabd admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:04.273532 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba745c5-aa34-41de-8412-020f57416a1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:04.274379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba745c5-aa34-41de-8412-020f57416a1e service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:04.274824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba745c5-aa34-41de-8412-020f57416a1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:04.275161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba745c5-aa34-41de-8412-020f57416a1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:04.286975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba745c5-aa34-41de-8412-020f57416a1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:04.288667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba745c5-aa34-41de-8412-020f57416a1e service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:04.299823 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1179/2359] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:17:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:17:04.915271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87a468-d01b-4598-87bf-3bdde4974dd3 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:04.917299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-4a87a468-d01b-4598-87bf-3bdde4974dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:04.943078 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87a468-d01b-4598-87bf-3bdde4974dd3 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:04.951416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2f907-ba55-4d36-94bc-95bdcf27523f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:04.953032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2f907-ba55-4d36-94bc-95bdcf27523f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:04.953747 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2f907-ba55-4d36-94bc-95bdcf27523f admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:04.954187 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2f907-ba55-4d36-94bc-95bdcf27523f admin admin] PATH_INFO: `/v3/projects/f5210ed64a7e4ac0927370738a21e149` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:04.956399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2f907-ba55-4d36-94bc-95bdcf27523f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5210ed64a7e4ac0927370738a21e149)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:04.959529 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2f907-ba55-4d36-94bc-95bdcf27523f admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:05.144600 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-96f2f907-ba55-4d36-94bc-95bdcf27523f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5210ed64a7e4ac0927370738a21e149', 'request_id': 'req-96f2f907-ba55-4d36-94bc-95bdcf27523f'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:05.153380 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-96f2f907-ba55-4d36-94bc-95bdcf27523f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5210ed64a7e4ac0927370738a21e149', 'request_id': 'req-96f2f907-ba55-4d36-94bc-95bdcf27523f'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:05.160499 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-96f2f907-ba55-4d36-94bc-95bdcf27523f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5210ed64a7e4ac0927370738a21e149', 'request_id': 'req-96f2f907-ba55-4d36-94bc-95bdcf27523f'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:05.249291 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-96f2f907-ba55-4d36-94bc-95bdcf27523f admin admin] The token cache is being invalidate because project f5210ed64a7e4ac0927370738a21e149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:17:05.249672 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-96f2f907-ba55-4d36-94bc-95bdcf27523f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:05.250436 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-96f2f907-ba55-4d36-94bc-95bdcf27523f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:05.252249 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1181/2360] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:17:04 2022] DELETE /identity/v3/projects/f5210ed64a7e4ac0927370738a21e149 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:17:05.257441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba745c5-aa34-41de-8412-020f57416a1e service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:05.269609 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba745c5-aa34-41de-8412-020f57416a1e service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:05.292561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:17:05.339058 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fafec-99ab-4c1d-b613-9e6797ea02e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:05.340107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fafec-99ab-4c1d-b613-9e6797ea02e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:05.340388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fafec-99ab-4c1d-b613-9e6797ea02e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:05.340663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fafec-99ab-4c1d-b613-9e6797ea02e3 admin admin] PATH_INFO: `/v3/users/0c345c9595f043d490a1db17ab08214c` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:05.342309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fafec-99ab-4c1d-b613-9e6797ea02e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c345c9595f043d490a1db17ab08214c)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:05.344524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fafec-99ab-4c1d-b613-9e6797ea02e3 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:05.583964 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ff2fafec-99ab-4c1d-b613-9e6797ea02e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c345c9595f043d490a1db17ab08214c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:05.601779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ff2fafec-99ab-4c1d-b613-9e6797ea02e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c345c9595f043d490a1db17ab08214c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:05.603378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ff2fafec-99ab-4c1d-b613-9e6797ea02e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c345c9595f043d490a1db17ab08214c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:05.623403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ff2fafec-99ab-4c1d-b613-9e6797ea02e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c345c9595f043d490a1db17ab08214c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:05.728745 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ff2fafec-99ab-4c1d-b613-9e6797ea02e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c345c9595f043d490a1db17ab08214c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:05.742221 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1180/2361] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:17:05 2022] DELETE /identity/v3/users/0c345c9595f043d490a1db17ab08214c => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:17:07.338378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2f907-ba55-4d36-94bc-95bdcf27523f admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:07.340322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-96f2f907-ba55-4d36-94bc-95bdcf27523f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:07.357651 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2f907-ba55-4d36-94bc-95bdcf27523f admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:07.446397 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65238b80-f292-4711-a819-1d217942dfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:07.447345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65238b80-f292-4711-a819-1d217942dfaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:07.447787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65238b80-f292-4711-a819-1d217942dfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:07.448233 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65238b80-f292-4711-a819-1d217942dfaf admin admin] PATH_INFO: `/v3/projects/4af97574abb245cba7280db88ac2e249` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:07.449662 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65238b80-f292-4711-a819-1d217942dfaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4af97574abb245cba7280db88ac2e249)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:07.451716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65238b80-f292-4711-a819-1d217942dfaf admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:07.587316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-65238b80-f292-4711-a819-1d217942dfaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4af97574abb245cba7280db88ac2e249', 'request_id': 'req-65238b80-f292-4711-a819-1d217942dfaf'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:07.590610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:17:07.597411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-65238b80-f292-4711-a819-1d217942dfaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4af97574abb245cba7280db88ac2e249', 'request_id': 'req-65238b80-f292-4711-a819-1d217942dfaf'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:07.605263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-65238b80-f292-4711-a819-1d217942dfaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4af97574abb245cba7280db88ac2e249', 'request_id': 'req-65238b80-f292-4711-a819-1d217942dfaf'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:07.706480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-65238b80-f292-4711-a819-1d217942dfaf admin admin] The token cache is being invalidate because project 4af97574abb245cba7280db88ac2e249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:17:07.706912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-65238b80-f292-4711-a819-1d217942dfaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:07.707699 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-65238b80-f292-4711-a819-1d217942dfaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:07.709765 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1182/2362] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:17:07 2022] DELETE /identity/v3/projects/4af97574abb245cba7280db88ac2e249 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:17:07.714887 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fafec-99ab-4c1d-b613-9e6797ea02e3 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:07.717134 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ff2fafec-99ab-4c1d-b613-9e6797ea02e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:07.733399 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fafec-99ab-4c1d-b613-9e6797ea02e3 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:07.841885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5aabf-3243-4a1e-a371-c6425dc46c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:07.843399 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5aabf-3243-4a1e-a371-c6425dc46c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:07.843921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5aabf-3243-4a1e-a371-c6425dc46c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:07.844438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5aabf-3243-4a1e-a371-c6425dc46c34 admin admin] PATH_INFO: `/v3/users/95ff51a7c9424982b5d63484b9918f61` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:07.849185 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5aabf-3243-4a1e-a371-c6425dc46c34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95ff51a7c9424982b5d63484b9918f61)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:07.867928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5aabf-3243-4a1e-a371-c6425dc46c34 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:08.098366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:17:08.102717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4ac5aabf-3243-4a1e-a371-c6425dc46c34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95ff51a7c9424982b5d63484b9918f61'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:08.122227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4ac5aabf-3243-4a1e-a371-c6425dc46c34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95ff51a7c9424982b5d63484b9918f61'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:08.123668 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4ac5aabf-3243-4a1e-a371-c6425dc46c34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95ff51a7c9424982b5d63484b9918f61'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:08.174852 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4ac5aabf-3243-4a1e-a371-c6425dc46c34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95ff51a7c9424982b5d63484b9918f61'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:08.332143 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4ac5aabf-3243-4a1e-a371-c6425dc46c34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95ff51a7c9424982b5d63484b9918f61'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:08.362754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1181/2363] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:17:07 2022] DELETE /identity/v3/users/95ff51a7c9424982b5d63484b9918f61 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:17:09.380156 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65238b80-f292-4711-a819-1d217942dfaf admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:09.385406 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-65238b80-f292-4711-a819-1d217942dfaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:09.423564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65238b80-f292-4711-a819-1d217942dfaf admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:09.504452 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debd554-f5df-443e-a347-73070923aa17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:09.505827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debd554-f5df-443e-a347-73070923aa17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:09.506109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debd554-f5df-443e-a347-73070923aa17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:09.507438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debd554-f5df-443e-a347-73070923aa17 admin admin] PATH_INFO: `/v3/projects/afaa3634d9aa49d984005cfdf964da95` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:09.509872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debd554-f5df-443e-a347-73070923aa17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afaa3634d9aa49d984005cfdf964da95)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:09.515878 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debd554-f5df-443e-a347-73070923aa17 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:09.711492 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5debd554-f5df-443e-a347-73070923aa17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afaa3634d9aa49d984005cfdf964da95', 'request_id': 'req-5debd554-f5df-443e-a347-73070923aa17'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:09.722434 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5debd554-f5df-443e-a347-73070923aa17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afaa3634d9aa49d984005cfdf964da95', 'request_id': 'req-5debd554-f5df-443e-a347-73070923aa17'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:09.731563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5debd554-f5df-443e-a347-73070923aa17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afaa3634d9aa49d984005cfdf964da95', 'request_id': 'req-5debd554-f5df-443e-a347-73070923aa17'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:09.859829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5debd554-f5df-443e-a347-73070923aa17 admin admin] The token cache is being invalidate because project afaa3634d9aa49d984005cfdf964da95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:17:09.860122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5debd554-f5df-443e-a347-73070923aa17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:09.860783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5debd554-f5df-443e-a347-73070923aa17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:09.862353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1183/2364] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:17:09 2022] DELETE /identity/v3/projects/afaa3634d9aa49d984005cfdf964da95 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:17:09.867609 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a0513-268f-4339-98e0-17abc42ad566 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:09.868184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a0513-268f-4339-98e0-17abc42ad566 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:09.868278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a0513-268f-4339-98e0-17abc42ad566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:09.891790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-496a0513-268f-4339-98e0-17abc42ad566 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:17:09.893070 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-496a0513-268f-4339-98e0-17abc42ad566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:09.958923 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1182/2365] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:17:09.968129 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9cf31-040c-47d3-8eba-892238602e9c None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:09.968408 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9cf31-040c-47d3-8eba-892238602e9c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:09.968694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9cf31-040c-47d3-8eba-892238602e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:09.989866 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-65b9cf31-040c-47d3-8eba-892238602e9c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:17:09.991299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-65b9cf31-040c-47d3-8eba-892238602e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:09.999548 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1184/2366] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:17:10.012269 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a0513-268f-4339-98e0-17abc42ad566 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:10.024127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a0513-268f-4339-98e0-17abc42ad566 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:10.029000 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239a957-7897-4d0c-89d5-858e31d65ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:10.029663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239a957-7897-4d0c-89d5-858e31d65ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:10.029953 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239a957-7897-4d0c-89d5-858e31d65ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:10.030233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239a957-7897-4d0c-89d5-858e31d65ef6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:10.031010 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239a957-7897-4d0c-89d5-858e31d65ef6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:17:10.031327 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239a957-7897-4d0c-89d5-858e31d65ef6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:10.032943 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239a957-7897-4d0c-89d5-858e31d65ef6 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:10.045376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1183/2367] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:17:10.050278 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9cf31-040c-47d3-8eba-892238602e9c None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:10.060172 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9cf31-040c-47d3-8eba-892238602e9c None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:10.064954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f09a8-35c6-4e53-9ce1-bf51060a8f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:10.065621 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f09a8-35c6-4e53-9ce1-bf51060a8f1e admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:10.065911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f09a8-35c6-4e53-9ce1-bf51060a8f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:10.066194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f09a8-35c6-4e53-9ce1-bf51060a8f1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:10.067135 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f09a8-35c6-4e53-9ce1-bf51060a8f1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:10.069611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f09a8-35c6-4e53-9ce1-bf51060a8f1e admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:10.291990 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1185/2368] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:17:10 2022] POST /identity/v3/projects => generated 379 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:17:10.293594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239a957-7897-4d0c-89d5-858e31d65ef6 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:10.315746 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239a957-7897-4d0c-89d5-858e31d65ef6 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:10.508254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625d62a-728c-47de-b60f-56ef1f05fd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:10.510058 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625d62a-728c-47de-b60f-56ef1f05fd1f admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:10.511056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625d62a-728c-47de-b60f-56ef1f05fd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:10.511979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625d62a-728c-47de-b60f-56ef1f05fd1f admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:10.514187 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625d62a-728c-47de-b60f-56ef1f05fd1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:10.521907 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625d62a-728c-47de-b60f-56ef1f05fd1f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:10.662224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1184/2369] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:17:10 2022] POST /identity/v3/users => generated 290 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:17:10.668076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f09a8-35c6-4e53-9ce1-bf51060a8f1e admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:10.692730 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f09a8-35c6-4e53-9ce1-bf51060a8f1e admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:10.711507 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbbaa3-8a5c-432c-ab42-0a585bcfd0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:10.714190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbbaa3-8a5c-432c-ab42-0a585bcfd0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:10.714995 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbbaa3-8a5c-432c-ab42-0a585bcfd0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:10.718286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbbaa3-8a5c-432c-ab42-0a585bcfd0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:10.720746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbbaa3-8a5c-432c-ab42-0a585bcfd0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:10.724767 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbbaa3-8a5c-432c-ab42-0a585bcfd0f5 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:10.752288 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1186/2370] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:17:10.760068 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625d62a-728c-47de-b60f-56ef1f05fd1f admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:10.797142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625d62a-728c-47de-b60f-56ef1f05fd1f admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:10.815247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597460c3-f84f-4396-a4a0-22216e628112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:10.815247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597460c3-f84f-4396-a4a0-22216e628112 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:10.815247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597460c3-f84f-4396-a4a0-22216e628112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:10.815750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597460c3-f84f-4396-a4a0-22216e628112 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:10.817702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597460c3-f84f-4396-a4a0-22216e628112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:10.824386 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597460c3-f84f-4396-a4a0-22216e628112 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:10.861822 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1185/2371] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:17:10.867474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbbaa3-8a5c-432c-ab42-0a585bcfd0f5 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:10.905548 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbbaa3-8a5c-432c-ab42-0a585bcfd0f5 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:10.916131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019a778-a40b-45e2-82f8-64a68c6736cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:10.917792 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019a778-a40b-45e2-82f8-64a68c6736cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:10.918804 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019a778-a40b-45e2-82f8-64a68c6736cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:10.919850 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019a778-a40b-45e2-82f8-64a68c6736cf admin admin] PATH_INFO: `/v3/projects/94513773e0c94187bbdaeb82c7304eae/users/6bfa2a00688d4eab993059ff176d8920/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:10.968857 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019a778-a40b-45e2-82f8-64a68c6736cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=94513773e0c94187bbdaeb82c7304eae, user_id=6bfa2a00688d4eab993059ff176d8920, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:10.980986 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019a778-a40b-45e2-82f8-64a68c6736cf admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:11.140079 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1187/2372] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:17:10 2022] PUT /identity/v3/projects/94513773e0c94187bbdaeb82c7304eae/users/6bfa2a00688d4eab993059ff176d8920/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:17:11.146514 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de509fc-c52a-4171-a276-2e9c16af1055 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:11.148871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de509fc-c52a-4171-a276-2e9c16af1055 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:11.151285 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de509fc-c52a-4171-a276-2e9c16af1055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:11.188533 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-1de509fc-c52a-4171-a276-2e9c16af1055 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:17:11.192014 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1de509fc-c52a-4171-a276-2e9c16af1055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:11.244466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:17:11.347484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1186/2373] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:17:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:17:11.362969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019a778-a40b-45e2-82f8-64a68c6736cf admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:11.365698 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-3019a778-a40b-45e2-82f8-64a68c6736cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:11.380385 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:17:11.461634 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019a778-a40b-45e2-82f8-64a68c6736cf admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:11.465887 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed97a6-fef4-4cd9-bdf2-0973749cabea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:11.469518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed97a6-fef4-4cd9-bdf2-0973749cabea service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:11.469518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed97a6-fef4-4cd9-bdf2-0973749cabea service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:11.469518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed97a6-fef4-4cd9-bdf2-0973749cabea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:11.479977 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed97a6-fef4-4cd9-bdf2-0973749cabea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:11.481934 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed97a6-fef4-4cd9-bdf2-0973749cabea service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:11.507049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1188/2374] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:17:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:17:14.680939 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de509fc-c52a-4171-a276-2e9c16af1055 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:14.682493 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1de509fc-c52a-4171-a276-2e9c16af1055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:14.719218 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de509fc-c52a-4171-a276-2e9c16af1055 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:14.735438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4437b-8ee2-4b6e-9fd1-383e35a6c35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:14.736502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4437b-8ee2-4b6e-9fd1-383e35a6c35b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:14.736956 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4437b-8ee2-4b6e-9fd1-383e35a6c35b admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:14.737586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4437b-8ee2-4b6e-9fd1-383e35a6c35b admin admin] PATH_INFO: `/v3/users/576d17874c2b4cebb8faaf7f19aac800` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:14.739466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4437b-8ee2-4b6e-9fd1-383e35a6c35b admin admin] RBAC: Authorizing `identity:delete_user(user_id=576d17874c2b4cebb8faaf7f19aac800)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:14.742392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4437b-8ee2-4b6e-9fd1-383e35a6c35b admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:15.105696 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-28e4437b-8ee2-4b6e-9fd1-383e35a6c35b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '576d17874c2b4cebb8faaf7f19aac800'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:15.125281 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-28e4437b-8ee2-4b6e-9fd1-383e35a6c35b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '576d17874c2b4cebb8faaf7f19aac800'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:15.126368 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-28e4437b-8ee2-4b6e-9fd1-383e35a6c35b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '576d17874c2b4cebb8faaf7f19aac800'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:15.158264 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-28e4437b-8ee2-4b6e-9fd1-383e35a6c35b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '576d17874c2b4cebb8faaf7f19aac800'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:15.257440 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-28e4437b-8ee2-4b6e-9fd1-383e35a6c35b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '576d17874c2b4cebb8faaf7f19aac800'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:15.268003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1187/2375] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:17:14 2022] DELETE /identity/v3/users/576d17874c2b4cebb8faaf7f19aac800 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:17:15.269885 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f78fdd-9da5-4175-b296-221712b2e0d0 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:15.270496 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f78fdd-9da5-4175-b296-221712b2e0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:15.270763 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f78fdd-9da5-4175-b296-221712b2e0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:15.332856 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d5f78fdd-9da5-4175-b296-221712b2e0d0 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:17:15.334423 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d5f78fdd-9da5-4175-b296-221712b2e0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:15.448540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1189/2376] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:17:15.474715 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4437b-8ee2-4b6e-9fd1-383e35a6c35b admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:15.476910 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-28e4437b-8ee2-4b6e-9fd1-383e35a6c35b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:15.592222 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4437b-8ee2-4b6e-9fd1-383e35a6c35b admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:15.597453 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0129f-20bc-4e40-b480-180d21fcc856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:15.598175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0129f-20bc-4e40-b480-180d21fcc856 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:15.598512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0129f-20bc-4e40-b480-180d21fcc856 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:15.598754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0129f-20bc-4e40-b480-180d21fcc856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:15.618139 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0129f-20bc-4e40-b480-180d21fcc856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:15.621238 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0129f-20bc-4e40-b480-180d21fcc856 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:15.652671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1188/2377] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:17:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:17:17.117361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f78fdd-9da5-4175-b296-221712b2e0d0 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:17.118925 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d5f78fdd-9da5-4175-b296-221712b2e0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:17.153555 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f78fdd-9da5-4175-b296-221712b2e0d0 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:17.172176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c0a35b-c4f6-4393-8d50-8025e73f4f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:17.180209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0a35b-c4f6-4393-8d50-8025e73f4f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:17.180209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0a35b-c4f6-4393-8d50-8025e73f4f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:17.180209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0a35b-c4f6-4393-8d50-8025e73f4f9d admin admin] PATH_INFO: `/v3/projects/693dbef8bddd4f49878d8b16b1d675fd` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:17.180209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c0a35b-c4f6-4393-8d50-8025e73f4f9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=693dbef8bddd4f49878d8b16b1d675fd)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:17.182146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c0a35b-c4f6-4393-8d50-8025e73f4f9d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:17.392409 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-07c0a35b-c4f6-4393-8d50-8025e73f4f9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '693dbef8bddd4f49878d8b16b1d675fd', 'request_id': 'req-07c0a35b-c4f6-4393-8d50-8025e73f4f9d'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:17.409822 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-07c0a35b-c4f6-4393-8d50-8025e73f4f9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '693dbef8bddd4f49878d8b16b1d675fd', 'request_id': 'req-07c0a35b-c4f6-4393-8d50-8025e73f4f9d'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:17.432026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-07c0a35b-c4f6-4393-8d50-8025e73f4f9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '693dbef8bddd4f49878d8b16b1d675fd', 'request_id': 'req-07c0a35b-c4f6-4393-8d50-8025e73f4f9d'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:17.639964 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-07c0a35b-c4f6-4393-8d50-8025e73f4f9d admin admin] The token cache is being invalidate because project 693dbef8bddd4f49878d8b16b1d675fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:17:17.640795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-07c0a35b-c4f6-4393-8d50-8025e73f4f9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:17.641867 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-07c0a35b-c4f6-4393-8d50-8025e73f4f9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:17:17.644234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1190/2378] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:17:17 2022] DELETE /identity/v3/projects/693dbef8bddd4f49878d8b16b1d675fd => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:17:17.651568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db77665-7b81-4b4d-a48f-de723be5268a None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:17.652343 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db77665-7b81-4b4d-a48f-de723be5268a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:17.653125 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db77665-7b81-4b4d-a48f-de723be5268a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:17.690038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-9db77665-7b81-4b4d-a48f-de723be5268a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:17:17.692682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-9db77665-7b81-4b4d-a48f-de723be5268a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:17.791916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:17:17.837648 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1189/2379] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:17:17.846860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d2a1d-ddc0-42aa-8c8d-501d2610690a None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:17.847192 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d2a1d-ddc0-42aa-8c8d-501d2610690a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:17.847411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d2a1d-ddc0-42aa-8c8d-501d2610690a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:17.871018 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-6e4d2a1d-ddc0-42aa-8c8d-501d2610690a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:17:17.872759 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6e4d2a1d-ddc0-42aa-8c8d-501d2610690a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:17.884229 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1191/2380] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:17:17.889588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db77665-7b81-4b4d-a48f-de723be5268a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:17.910960 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db77665-7b81-4b4d-a48f-de723be5268a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:17.920316 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7a42c-2eba-4285-8480-4f568a400c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:17.921562 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7a42c-2eba-4285-8480-4f568a400c65 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:17.921973 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7a42c-2eba-4285-8480-4f568a400c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:17.922394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7a42c-2eba-4285-8480-4f568a400c65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:17.923365 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7a42c-2eba-4285-8480-4f568a400c65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:17:17.923805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7a42c-2eba-4285-8480-4f568a400c65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:17.926258 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7a42c-2eba-4285-8480-4f568a400c65 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:17.950840 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1190/2381] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:17:17.956451 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d2a1d-ddc0-42aa-8c8d-501d2610690a None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:17.969849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d2a1d-ddc0-42aa-8c8d-501d2610690a None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:17.975553 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e83fcb-fafd-4155-89a5-6de9554a53ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:17.976296 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e83fcb-fafd-4155-89a5-6de9554a53ce admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:17.976589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e83fcb-fafd-4155-89a5-6de9554a53ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:17.976867 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e83fcb-fafd-4155-89a5-6de9554a53ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:17.977747 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e83fcb-fafd-4155-89a5-6de9554a53ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:17.979884 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e83fcb-fafd-4155-89a5-6de9554a53ce admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:18.135799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1192/2382] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:17:17 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:17:18.145405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7a42c-2eba-4285-8480-4f568a400c65 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:18.183927 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7a42c-2eba-4285-8480-4f568a400c65 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:18.295885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb0c35-1850-4af6-9068-9725e1e349d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:18.296718 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb0c35-1850-4af6-9068-9725e1e349d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:18.297058 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb0c35-1850-4af6-9068-9725e1e349d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:18.297246 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb0c35-1850-4af6-9068-9725e1e349d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:18.297971 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb0c35-1850-4af6-9068-9725e1e349d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:18.300572 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb0c35-1850-4af6-9068-9725e1e349d2 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:18.535387 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4959a3-1a96-478b-8adc-f0eddfb0138a None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:18.536170 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4959a3-1a96-478b-8adc-f0eddfb0138a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:18.536869 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4959a3-1a96-478b-8adc-f0eddfb0138a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:18.545866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1191/2383] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:17:18 2022] POST /identity/v3/users => generated 292 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:17:18.547691 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:17:18.551718 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb0c35-1850-4af6-9068-9725e1e349d2 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:18.562162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-ad4959a3-1a96-478b-8adc-f0eddfb0138a None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:17:18.563909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ad4959a3-1a96-478b-8adc-f0eddfb0138a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:18.580420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb0c35-1850-4af6-9068-9725e1e349d2 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:18.598689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51617e47-51fe-47af-ad6b-8344df09bd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:18.599545 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51617e47-51fe-47af-ad6b-8344df09bd62 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:18.599866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51617e47-51fe-47af-ad6b-8344df09bd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:18.600157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51617e47-51fe-47af-ad6b-8344df09bd62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:18.602411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51617e47-51fe-47af-ad6b-8344df09bd62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:18.621400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51617e47-51fe-47af-ad6b-8344df09bd62 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:18.630632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1192/2384] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:17:18.635392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51617e47-51fe-47af-ad6b-8344df09bd62 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:18.647364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51617e47-51fe-47af-ad6b-8344df09bd62 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:18.653043 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0256fb-1710-41aa-86a8-f1400af2d71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:18.654306 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0256fb-1710-41aa-86a8-f1400af2d71d admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:18.654830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0256fb-1710-41aa-86a8-f1400af2d71d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:18.655206 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0256fb-1710-41aa-86a8-f1400af2d71d admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:18.656031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0256fb-1710-41aa-86a8-f1400af2d71d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:18.658114 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0256fb-1710-41aa-86a8-f1400af2d71d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:18.679879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1193/2385] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:17:18.684462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0256fb-1710-41aa-86a8-f1400af2d71d admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:18.714655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0256fb-1710-41aa-86a8-f1400af2d71d admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:18.723359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1193/2386] 10.222.0.83 () {58 vars in 967 bytes} [Mon Jul 18 15:17:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:17:18.731402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475e003-0ef6-48b8-997f-be3d37aec005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:18.732253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475e003-0ef6-48b8-997f-be3d37aec005 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:18.732599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475e003-0ef6-48b8-997f-be3d37aec005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:18.732961 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475e003-0ef6-48b8-997f-be3d37aec005 admin admin] PATH_INFO: `/v3/projects/fcd59138153e461bb239770cc1bbc1f1/users/4746f061cfd148609ffaf568f869d6d8/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:18.738742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4959a3-1a96-478b-8adc-f0eddfb0138a None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:18.740200 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ad4959a3-1a96-478b-8adc-f0eddfb0138a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:18.763630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475e003-0ef6-48b8-997f-be3d37aec005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd59138153e461bb239770cc1bbc1f1, user_id=4746f061cfd148609ffaf568f869d6d8, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:18.782085 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475e003-0ef6-48b8-997f-be3d37aec005 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:18.848935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4959a3-1a96-478b-8adc-f0eddfb0138a None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:18.868435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cef22c-8cd2-4c70-aaf1-8b30a827700f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:18.868435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cef22c-8cd2-4c70-aaf1-8b30a827700f service swift] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:18.868435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cef22c-8cd2-4c70-aaf1-8b30a827700f service swift] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:18.868435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cef22c-8cd2-4c70-aaf1-8b30a827700f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:18.876954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cef22c-8cd2-4c70-aaf1-8b30a827700f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:18.879055 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cef22c-8cd2-4c70-aaf1-8b30a827700f service swift] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:18.890695 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1194/2387] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:17:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 15:17:18.908981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1194/2388] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:17:18 2022] PUT /identity/v3/projects/fcd59138153e461bb239770cc1bbc1f1/users/4746f061cfd148609ffaf568f869d6d8/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:17:18.919256 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a1b17-c30e-4650-bc38-dc29c9248de2 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:18.919564 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a1b17-c30e-4650-bc38-dc29c9248de2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:18.919845 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a1b17-c30e-4650-bc38-dc29c9248de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:18.964574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-146a1b17-c30e-4650-bc38-dc29c9248de2 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:17:18.965922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-146a1b17-c30e-4650-bc38-dc29c9248de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:19.086956 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1195/2389] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:17:19.105679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475e003-0ef6-48b8-997f-be3d37aec005 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:19.108842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b475e003-0ef6-48b8-997f-be3d37aec005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:19.285215 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475e003-0ef6-48b8-997f-be3d37aec005 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:19.294290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ccb5e-fb4a-441f-b654-ed5105981b71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:19.295408 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ccb5e-fb4a-441f-b654-ed5105981b71 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:19.295856 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ccb5e-fb4a-441f-b654-ed5105981b71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:19.296613 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ccb5e-fb4a-441f-b654-ed5105981b71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:19.316106 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ccb5e-fb4a-441f-b654-ed5105981b71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:19.324514 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ccb5e-fb4a-441f-b654-ed5105981b71 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:19.345960 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1195/2390] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:17:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:17:22.140180 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cef307-8b01-4906-b9f8-865a488f0dfb None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:22.141081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cef307-8b01-4906-b9f8-865a488f0dfb None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:22.141081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cef307-8b01-4906-b9f8-865a488f0dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:22.229947 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-73cef307-8b01-4906-b9f8-865a488f0dfb None None] MFA Rules not processed for user `4746f061cfd148609ffaf568f869d6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:17:22.233251 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-73cef307-8b01-4906-b9f8-865a488f0dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:22.329282 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:17:22.419635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1196/2391] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:17:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:17:22.449194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ccb5e-fb4a-441f-b654-ed5105981b71 service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:22.452958 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-8a2ccb5e-fb4a-441f-b654-ed5105981b71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:22.503652 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:17:22.621728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ccb5e-fb4a-441f-b654-ed5105981b71 service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:22.625998 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f1f9d-68a3-4c27-9af9-11ef5b86680e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:22.627063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f1f9d-68a3-4c27-9af9-11ef5b86680e service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:22.627567 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f1f9d-68a3-4c27-9af9-11ef5b86680e service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:22.627931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f1f9d-68a3-4c27-9af9-11ef5b86680e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:22.643067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f1f9d-68a3-4c27-9af9-11ef5b86680e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:22.652077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f1f9d-68a3-4c27-9af9-11ef5b86680e service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:22.667245 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1196/2392] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:17:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:17:23.876523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad14618-0aa2-48b2-8d73-4e83325ec261 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:23.877098 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad14618-0aa2-48b2-8d73-4e83325ec261 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:23.877098 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad14618-0aa2-48b2-8d73-4e83325ec261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:23.911931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-2ad14618-0aa2-48b2-8d73-4e83325ec261 None None] MFA Rules not processed for user `6bfa2a00688d4eab993059ff176d8920`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:17:23.911931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2ad14618-0aa2-48b2-8d73-4e83325ec261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:24.003798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1197/2393] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:17:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:17:24.009353 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f1f9d-68a3-4c27-9af9-11ef5b86680e service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:24.011033 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3e2f1f9d-68a3-4c27-9af9-11ef5b86680e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:24.029849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f1f9d-68a3-4c27-9af9-11ef5b86680e service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:24.036319 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8d458-c3dd-439c-bfff-a9981ae303ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:24.037018 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8d458-c3dd-439c-bfff-a9981ae303ec admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:24.037305 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8d458-c3dd-439c-bfff-a9981ae303ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:24.037734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8d458-c3dd-439c-bfff-a9981ae303ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:24.038470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8d458-c3dd-439c-bfff-a9981ae303ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:24.040477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8d458-c3dd-439c-bfff-a9981ae303ec admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:24.157572 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1197/2394] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:17:24 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:17:24.161645 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad14618-0aa2-48b2-8d73-4e83325ec261 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:24.180219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad14618-0aa2-48b2-8d73-4e83325ec261 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:24.281661 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629013b8-3b44-47cf-8c97-b19d72ca3c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:24.282711 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629013b8-3b44-47cf-8c97-b19d72ca3c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:24.283206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629013b8-3b44-47cf-8c97-b19d72ca3c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:24.283485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629013b8-3b44-47cf-8c97-b19d72ca3c61 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:24.284204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629013b8-3b44-47cf-8c97-b19d72ca3c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:24.286279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629013b8-3b44-47cf-8c97-b19d72ca3c61 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:24.529093 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1198/2395] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:17:24 2022] POST /identity/v3/users => generated 295 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:17:24.533560 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8d458-c3dd-439c-bfff-a9981ae303ec admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:24.550389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8d458-c3dd-439c-bfff-a9981ae303ec admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:24.557187 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f7900-e10c-4f0a-a8b7-bb28d0fbc1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:24.557909 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f7900-e10c-4f0a-a8b7-bb28d0fbc1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:24.558200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f7900-e10c-4f0a-a8b7-bb28d0fbc1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:24.558504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f7900-e10c-4f0a-a8b7-bb28d0fbc1e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:24.559602 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f7900-e10c-4f0a-a8b7-bb28d0fbc1e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:24.562123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f7900-e10c-4f0a-a8b7-bb28d0fbc1e2 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:24.579040 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1198/2396] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:17:24.585798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629013b8-3b44-47cf-8c97-b19d72ca3c61 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:24.605391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629013b8-3b44-47cf-8c97-b19d72ca3c61 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:24.614008 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b84b9-4555-4153-96ec-0fa984c11d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:24.616938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b84b9-4555-4153-96ec-0fa984c11d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:24.617565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b84b9-4555-4153-96ec-0fa984c11d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:24.618078 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b84b9-4555-4153-96ec-0fa984c11d7d admin admin] PATH_INFO: `/v3/projects/57feb941d284478fa86bbc3166240b61/users/eaf034a79b434d0a9f79c76b0b8fcc2c/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:24.648279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b84b9-4555-4153-96ec-0fa984c11d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=57feb941d284478fa86bbc3166240b61, user_id=eaf034a79b434d0a9f79c76b0b8fcc2c, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:24.653912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b84b9-4555-4153-96ec-0fa984c11d7d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:24.735387 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1199/2397] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:17:24 2022] PUT /identity/v3/projects/57feb941d284478fa86bbc3166240b61/users/eaf034a79b434d0a9f79c76b0b8fcc2c/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:17:24.740188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f7900-e10c-4f0a-a8b7-bb28d0fbc1e2 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:24.759978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f7900-e10c-4f0a-a8b7-bb28d0fbc1e2 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:24.844127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342a703-a0b2-47a1-8016-06d646dc657d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:24.844682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342a703-a0b2-47a1-8016-06d646dc657d admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:24.844968 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342a703-a0b2-47a1-8016-06d646dc657d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:24.845249 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342a703-a0b2-47a1-8016-06d646dc657d admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:24.848637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342a703-a0b2-47a1-8016-06d646dc657d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:24.848637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342a703-a0b2-47a1-8016-06d646dc657d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:24.863480 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1199/2398] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:17:24.868781 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b84b9-4555-4153-96ec-0fa984c11d7d admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:24.884279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b84b9-4555-4153-96ec-0fa984c11d7d admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:24.889684 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1eb9fa-9574-46b6-b05c-37083b02d266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:24.895847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1eb9fa-9574-46b6-b05c-37083b02d266 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:24.895847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1eb9fa-9574-46b6-b05c-37083b02d266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:24.895847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1eb9fa-9574-46b6-b05c-37083b02d266 admin admin] PATH_INFO: `/v3/projects/57feb941d284478fa86bbc3166240b61/users/eaf034a79b434d0a9f79c76b0b8fcc2c/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:24.895847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1eb9fa-9574-46b6-b05c-37083b02d266 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57feb941d284478fa86bbc3166240b61, user_id=eaf034a79b434d0a9f79c76b0b8fcc2c, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:24.896579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1eb9fa-9574-46b6-b05c-37083b02d266 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:24.909674 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1200/2399] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:17:24 2022] PUT /identity/v3/projects/57feb941d284478fa86bbc3166240b61/users/eaf034a79b434d0a9f79c76b0b8fcc2c/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:17:26.096833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222830c-4238-4e61-b875-bbe1a71fb8f8 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:26.098056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222830c-4238-4e61-b875-bbe1a71fb8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:26.111000 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222830c-4238-4e61-b875-bbe1a71fb8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:26.122586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-a222830c-4238-4e61-b875-bbe1a71fb8f8 None None] MFA Rules not processed for user `4746f061cfd148609ffaf568f869d6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:17:26.123895 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a222830c-4238-4e61-b875-bbe1a71fb8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:26.205190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1200/2400] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:17:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:17:26.507910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb5773-4ffd-4736-af0f-546aeca25450 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:26.509060 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb5773-4ffd-4736-af0f-546aeca25450 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:26.509506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb5773-4ffd-4736-af0f-546aeca25450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:26.544771 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-eefb5773-4ffd-4736-af0f-546aeca25450 None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:17:26.546411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-eefb5773-4ffd-4736-af0f-546aeca25450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:26.557553 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:17:26.751823 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1201/2401] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Jul 18 15:17:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:17:26.802378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222830c-4238-4e61-b875-bbe1a71fb8f8 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:26.889588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:17:27.002427 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222830c-4238-4e61-b875-bbe1a71fb8f8 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:27.024104 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f0351-966e-459a-be2c-12cd8547dcc9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:27.024603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f0351-966e-459a-be2c-12cd8547dcc9 service swift] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:27.024952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f0351-966e-459a-be2c-12cd8547dcc9 service swift] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:27.026728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f0351-966e-459a-be2c-12cd8547dcc9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:27.053495 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f0351-966e-459a-be2c-12cd8547dcc9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:27.055800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f0351-966e-459a-be2c-12cd8547dcc9 service swift] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:27.079765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1201/2402] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:17:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 15:17:35.527707 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb5773-4ffd-4736-af0f-546aeca25450 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:35.532019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-eefb5773-4ffd-4736-af0f-546aeca25450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:35.642573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb5773-4ffd-4736-af0f-546aeca25450 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:35.651935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80c421-4fd1-4c76-85f6-19800b8e608e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:35.653205 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80c421-4fd1-4c76-85f6-19800b8e608e service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:35.653661 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80c421-4fd1-4c76-85f6-19800b8e608e service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:35.654294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80c421-4fd1-4c76-85f6-19800b8e608e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:35.658606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2b80c421-4fd1-4c76-85f6-19800b8e608e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:35.854859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80c421-4fd1-4c76-85f6-19800b8e608e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:35.856368 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80c421-4fd1-4c76-85f6-19800b8e608e service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:35.886584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1202/2403] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:17:35 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:17:37.613975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=20 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:17:39.287496 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8514f6-3458-49dc-8dd0-f812183fbd91 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:39.289632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8514f6-3458-49dc-8dd0-f812183fbd91 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:39.290211 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8514f6-3458-49dc-8dd0-f812183fbd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:39.368415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-3d8514f6-3458-49dc-8dd0-f812183fbd91 None None] MFA Rules not processed for user `eaf034a79b434d0a9f79c76b0b8fcc2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:17:39.374716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3d8514f6-3458-49dc-8dd0-f812183fbd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:39.523453 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1202/2404] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:17:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:17:39.554055 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80c421-4fd1-4c76-85f6-19800b8e608e service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:17:39.554055 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2b80c421-4fd1-4c76-85f6-19800b8e608e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:17:39.779472 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:17:39.790158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80c421-4fd1-4c76-85f6-19800b8e608e service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:17:39.804348 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06947d17-ba6b-4a49-ad02-b900b30922e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:17:39.805085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06947d17-ba6b-4a49-ad02-b900b30922e3 service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:17:39.805378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06947d17-ba6b-4a49-ad02-b900b30922e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:17:39.805661 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06947d17-ba6b-4a49-ad02-b900b30922e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:17:40.016673 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06947d17-ba6b-4a49-ad02-b900b30922e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:17:40.022951 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06947d17-ba6b-4a49-ad02-b900b30922e3 service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:17:40.052341 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1203/2405] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:17:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:17:49.517357 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=34 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:17:50.035248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=26 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:18:00.825522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c06ffa-d41b-46ef-aad1-8c66a42693cb None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:00.826503 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c06ffa-d41b-46ef-aad1-8c66a42693cb None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:00.826503 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c06ffa-d41b-46ef-aad1-8c66a42693cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:00.846586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-e9c06ffa-d41b-46ef-aad1-8c66a42693cb None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:18:00.848079 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e9c06ffa-d41b-46ef-aad1-8c66a42693cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:00.855930 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1203/2406] 10.222.0.83 () {58 vars in 967 bytes} [Mon Jul 18 15:18:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:18:00.870520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06947d17-ba6b-4a49-ad02-b900b30922e3 service nova] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:00.883136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06947d17-ba6b-4a49-ad02-b900b30922e3 service nova] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:00.888578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d16ffd-b3f6-4be8-81e7-7fdc3d12f73b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:00.889235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d16ffd-b3f6-4be8-81e7-7fdc3d12f73b service swift] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:00.889525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d16ffd-b3f6-4be8-81e7-7fdc3d12f73b service swift] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:00.889808 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d16ffd-b3f6-4be8-81e7-7fdc3d12f73b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:00.899087 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d16ffd-b3f6-4be8-81e7-7fdc3d12f73b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:00.900676 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d16ffd-b3f6-4be8-81e7-7fdc3d12f73b service swift] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:00.909208 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1204/2407] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:18:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 15:18:04.424221 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c06ffa-d41b-46ef-aad1-8c66a42693cb None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:04.426786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e9c06ffa-d41b-46ef-aad1-8c66a42693cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:04.500605 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c06ffa-d41b-46ef-aad1-8c66a42693cb None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:04.505926 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cd566-b909-4534-8079-fd36efd54876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:04.507765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cd566-b909-4534-8079-fd36efd54876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:04.508053 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cd566-b909-4534-8079-fd36efd54876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:04.508336 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cd566-b909-4534-8079-fd36efd54876 admin admin] PATH_INFO: `/v3/users/faa3eace09844298a4cf0373ddc74de1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:04.509679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cd566-b909-4534-8079-fd36efd54876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faa3eace09844298a4cf0373ddc74de1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:04.511741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cd566-b909-4534-8079-fd36efd54876 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:04.711655 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4d8cd566-b909-4534-8079-fd36efd54876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faa3eace09844298a4cf0373ddc74de1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:04.719977 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4d8cd566-b909-4534-8079-fd36efd54876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faa3eace09844298a4cf0373ddc74de1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:04.720841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4d8cd566-b909-4534-8079-fd36efd54876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faa3eace09844298a4cf0373ddc74de1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:04.735363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4d8cd566-b909-4534-8079-fd36efd54876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faa3eace09844298a4cf0373ddc74de1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:04.821048 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4d8cd566-b909-4534-8079-fd36efd54876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faa3eace09844298a4cf0373ddc74de1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:04.827871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1204/2408] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:18:04 2022] DELETE /identity/v3/users/faa3eace09844298a4cf0373ddc74de1 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:18:05.126855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d16ffd-b3f6-4be8-81e7-7fdc3d12f73b service swift] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:05.140553 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d16ffd-b3f6-4be8-81e7-7fdc3d12f73b service swift] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:05.230125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cf46a-007b-4f32-ae61-95f48170e966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:05.231042 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cf46a-007b-4f32-ae61-95f48170e966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:05.231329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cf46a-007b-4f32-ae61-95f48170e966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:05.231603 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cf46a-007b-4f32-ae61-95f48170e966 admin admin] PATH_INFO: `/v3/projects/43a5c5beef1a42f0ac47a1f540e55c21` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:05.232856 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cf46a-007b-4f32-ae61-95f48170e966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43a5c5beef1a42f0ac47a1f540e55c21)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:05.234851 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cf46a-007b-4f32-ae61-95f48170e966 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:05.353219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-721cf46a-007b-4f32-ae61-95f48170e966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43a5c5beef1a42f0ac47a1f540e55c21', 'request_id': 'req-721cf46a-007b-4f32-ae61-95f48170e966'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:05.360100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-721cf46a-007b-4f32-ae61-95f48170e966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43a5c5beef1a42f0ac47a1f540e55c21', 'request_id': 'req-721cf46a-007b-4f32-ae61-95f48170e966'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:05.367543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-721cf46a-007b-4f32-ae61-95f48170e966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43a5c5beef1a42f0ac47a1f540e55c21', 'request_id': 'req-721cf46a-007b-4f32-ae61-95f48170e966'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:05.420784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-721cf46a-007b-4f32-ae61-95f48170e966 admin admin] The token cache is being invalidate because project 43a5c5beef1a42f0ac47a1f540e55c21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:18:05.421106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-721cf46a-007b-4f32-ae61-95f48170e966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:05.421727 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-721cf46a-007b-4f32-ae61-95f48170e966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:05.423389 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1205/2409] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:18:05 2022] DELETE /identity/v3/projects/43a5c5beef1a42f0ac47a1f540e55c21 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:18:05.429599 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47c647-4946-47be-ad28-a8a01477dca9 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:05.430363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47c647-4946-47be-ad28-a8a01477dca9 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:05.430363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47c647-4946-47be-ad28-a8a01477dca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:05.456820 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-ae47c647-4946-47be-ad28-a8a01477dca9 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:18:05.458120 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ae47c647-4946-47be-ad28-a8a01477dca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:05.527229 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1205/2410] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:18:05.535351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3191f68-0dc7-4b71-9e63-72aa684c60c7 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:05.535643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3191f68-0dc7-4b71-9e63-72aa684c60c7 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:05.535930 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3191f68-0dc7-4b71-9e63-72aa684c60c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:05.556967 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-f3191f68-0dc7-4b71-9e63-72aa684c60c7 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:18:05.558274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-f3191f68-0dc7-4b71-9e63-72aa684c60c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:05.567242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1206/2411] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:18:05.572479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47c647-4946-47be-ad28-a8a01477dca9 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:05.583128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47c647-4946-47be-ad28-a8a01477dca9 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:05.589802 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5d8f5-74b8-41e7-9d6e-8a31f9d33bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:05.590522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5d8f5-74b8-41e7-9d6e-8a31f9d33bea admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:05.591080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5d8f5-74b8-41e7-9d6e-8a31f9d33bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:05.591369 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5d8f5-74b8-41e7-9d6e-8a31f9d33bea admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:05.592150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5d8f5-74b8-41e7-9d6e-8a31f9d33bea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:18:05.592487 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5d8f5-74b8-41e7-9d6e-8a31f9d33bea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:05.594413 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5d8f5-74b8-41e7-9d6e-8a31f9d33bea admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:05.612568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:18:05.621972 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1206/2412] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:18:05.636095 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3191f68-0dc7-4b71-9e63-72aa684c60c7 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:05.654617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3191f68-0dc7-4b71-9e63-72aa684c60c7 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:05.662974 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691052f-e2eb-4db6-9608-21cb6f7e6650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:05.664258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691052f-e2eb-4db6-9608-21cb6f7e6650 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:05.664767 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691052f-e2eb-4db6-9608-21cb6f7e6650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:05.665264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691052f-e2eb-4db6-9608-21cb6f7e6650 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:05.666664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691052f-e2eb-4db6-9608-21cb6f7e6650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:05.670854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691052f-e2eb-4db6-9608-21cb6f7e6650 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:05.799026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1207/2413] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:18:05 2022] POST /identity/v3/projects => generated 397 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:18:05.804126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5d8f5-74b8-41e7-9d6e-8a31f9d33bea admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:05.815433 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5d8f5-74b8-41e7-9d6e-8a31f9d33bea admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:05.878154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d4cff-8641-4da5-b2b5-b268a1692054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:05.878774 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d4cff-8641-4da5-b2b5-b268a1692054 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:05.879065 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d4cff-8641-4da5-b2b5-b268a1692054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:05.879344 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d4cff-8641-4da5-b2b5-b268a1692054 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:05.880051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d4cff-8641-4da5-b2b5-b268a1692054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:05.882581 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d4cff-8641-4da5-b2b5-b268a1692054 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:05.974366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1207/2414] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:18:05 2022] POST /identity/v3/users => generated 299 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:18:05.979656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691052f-e2eb-4db6-9608-21cb6f7e6650 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:05.994426 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691052f-e2eb-4db6-9608-21cb6f7e6650 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:06.008335 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0444d9-5bdb-4b46-a3c1-36357780971b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:06.009238 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0444d9-5bdb-4b46-a3c1-36357780971b admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:06.009535 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0444d9-5bdb-4b46-a3c1-36357780971b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:06.010103 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0444d9-5bdb-4b46-a3c1-36357780971b admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:06.011035 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0444d9-5bdb-4b46-a3c1-36357780971b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:06.014416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0444d9-5bdb-4b46-a3c1-36357780971b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:06.041181 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1208/2415] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:18:06.042365 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d4cff-8641-4da5-b2b5-b268a1692054 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:06.056462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d4cff-8641-4da5-b2b5-b268a1692054 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:06.064043 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42508ff6-b654-4f8d-928e-ed23d0569fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:06.065037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42508ff6-b654-4f8d-928e-ed23d0569fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:06.065388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42508ff6-b654-4f8d-928e-ed23d0569fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:06.065745 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42508ff6-b654-4f8d-928e-ed23d0569fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:06.066629 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42508ff6-b654-4f8d-928e-ed23d0569fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:06.068373 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42508ff6-b654-4f8d-928e-ed23d0569fdc admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:06.082639 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1208/2416] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:18:06.088761 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0444d9-5bdb-4b46-a3c1-36357780971b admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:06.102857 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0444d9-5bdb-4b46-a3c1-36357780971b admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:06.107815 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b741aa-f8fc-4f41-9af0-d67420f709ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:06.108757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b741aa-f8fc-4f41-9af0-d67420f709ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:06.109070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b741aa-f8fc-4f41-9af0-d67420f709ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:06.109351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b741aa-f8fc-4f41-9af0-d67420f709ce admin admin] PATH_INFO: `/v3/projects/624f5188238c41ee956fd9d2eec8c9c8/users/27f01d6995f844f4bc9396a2cba4d8a6/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:06.128812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b741aa-f8fc-4f41-9af0-d67420f709ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=624f5188238c41ee956fd9d2eec8c9c8, user_id=27f01d6995f844f4bc9396a2cba4d8a6, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:06.132664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b741aa-f8fc-4f41-9af0-d67420f709ce admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:06.242894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1209/2417] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:18:06 2022] PUT /identity/v3/projects/624f5188238c41ee956fd9d2eec8c9c8/users/27f01d6995f844f4bc9396a2cba4d8a6/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:18:06.254183 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ea90c-4f1d-4ea8-9cd3-8482a18245b9 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:06.254483 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ea90c-4f1d-4ea8-9cd3-8482a18245b9 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:06.254787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ea90c-4f1d-4ea8-9cd3-8482a18245b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:06.291963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-ba3ea90c-4f1d-4ea8-9cd3-8482a18245b9 None None] MFA Rules not processed for user `27f01d6995f844f4bc9396a2cba4d8a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:18:06.293299 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ba3ea90c-4f1d-4ea8-9cd3-8482a18245b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:06.370557 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1209/2418] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:18:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:18:06.387515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b741aa-f8fc-4f41-9af0-d67420f709ce admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:06.389356 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-73b741aa-f8fc-4f41-9af0-d67420f709ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:06.405266 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:18:06.451455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b741aa-f8fc-4f41-9af0-d67420f709ce admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:06.455331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff5c4e-252e-40ce-9264-4587cd46eb4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:06.455990 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff5c4e-252e-40ce-9264-4587cd46eb4e service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:06.456273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff5c4e-252e-40ce-9264-4587cd46eb4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:06.456551 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff5c4e-252e-40ce-9264-4587cd46eb4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:06.466839 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff5c4e-252e-40ce-9264-4587cd46eb4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:06.468572 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff5c4e-252e-40ce-9264-4587cd46eb4e service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:06.483508 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1210/2419] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:18:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:18:15.852521 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa0edc-5930-4b6a-b197-3816367653ca None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:15.853479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa0edc-5930-4b6a-b197-3816367653ca None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:15.853479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa0edc-5930-4b6a-b197-3816367653ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:15.874850 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-f5fa0edc-5930-4b6a-b197-3816367653ca None None] MFA Rules not processed for user `648c869550ee437a8e775e997c741e58`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:18:15.877039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f5fa0edc-5930-4b6a-b197-3816367653ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:15.945019 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1210/2420] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:18:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:18:16.393766 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b2246-e7ae-4cc6-b7cc-f443cdac6cea None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:16.395240 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b2246-e7ae-4cc6-b7cc-f443cdac6cea None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:16.395698 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b2246-e7ae-4cc6-b7cc-f443cdac6cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:16.424932 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-653b2246-e7ae-4cc6-b7cc-f443cdac6cea None None] MFA Rules not processed for user `6bfa2a00688d4eab993059ff176d8920`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:18:16.428371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-653b2246-e7ae-4cc6-b7cc-f443cdac6cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:16.528463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1211/2421] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:18:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:18:16.776110 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa0edc-5930-4b6a-b197-3816367653ca None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:16.792345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:18:16.879238 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa0edc-5930-4b6a-b197-3816367653ca None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:16.890099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d03af-fb2d-4e2f-8589-5d2352322333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:16.891778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d03af-fb2d-4e2f-8589-5d2352322333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:16.892295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d03af-fb2d-4e2f-8589-5d2352322333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:16.892814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d03af-fb2d-4e2f-8589-5d2352322333 admin admin] PATH_INFO: `/v3/users/27f01d6995f844f4bc9396a2cba4d8a6` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:16.895513 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d03af-fb2d-4e2f-8589-5d2352322333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27f01d6995f844f4bc9396a2cba4d8a6)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:16.899048 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d03af-fb2d-4e2f-8589-5d2352322333 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:17.192891 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ea9d03af-fb2d-4e2f-8589-5d2352322333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27f01d6995f844f4bc9396a2cba4d8a6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:17.208937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ea9d03af-fb2d-4e2f-8589-5d2352322333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27f01d6995f844f4bc9396a2cba4d8a6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:17.210163 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ea9d03af-fb2d-4e2f-8589-5d2352322333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27f01d6995f844f4bc9396a2cba4d8a6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:17.255034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ea9d03af-fb2d-4e2f-8589-5d2352322333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27f01d6995f844f4bc9396a2cba4d8a6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:17.323538 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-ea9d03af-fb2d-4e2f-8589-5d2352322333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27f01d6995f844f4bc9396a2cba4d8a6'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:17.346680 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1211/2422] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:18:16 2022] DELETE /identity/v3/users/27f01d6995f844f4bc9396a2cba4d8a6 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:18:17.361123 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79a954-df15-4e89-b4ed-8825d542aaed None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:17.361527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79a954-df15-4e89-b4ed-8825d542aaed None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:17.361768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79a954-df15-4e89-b4ed-8825d542aaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:17.386265 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-aa79a954-df15-4e89-b4ed-8825d542aaed None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:18:17.388584 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-aa79a954-df15-4e89-b4ed-8825d542aaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:17.474747 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1212/2423] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:18:17.491865 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d03af-fb2d-4e2f-8589-5d2352322333 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:17.497502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ea9d03af-fb2d-4e2f-8589-5d2352322333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:17.582260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d03af-fb2d-4e2f-8589-5d2352322333 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:17.592753 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1bc94-b433-4dc7-8ded-39c35191cbdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:17.593785 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1bc94-b433-4dc7-8ded-39c35191cbdd service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:17.594059 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1bc94-b433-4dc7-8ded-39c35191cbdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:17.594342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1bc94-b433-4dc7-8ded-39c35191cbdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:17.606728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1bc94-b433-4dc7-8ded-39c35191cbdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:17.608700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1bc94-b433-4dc7-8ded-39c35191cbdd service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:17.625435 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1212/2424] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:18:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:18:18.345543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79a954-df15-4e89-b4ed-8825d542aaed None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:18.347384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-aa79a954-df15-4e89-b4ed-8825d542aaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:18.361421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79a954-df15-4e89-b4ed-8825d542aaed None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:18.369606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ba7a8-6a9f-4f6a-a851-64e1c23df7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:18.371194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ba7a8-6a9f-4f6a-a851-64e1c23df7ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:18.371755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ba7a8-6a9f-4f6a-a851-64e1c23df7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:18.372273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ba7a8-6a9f-4f6a-a851-64e1c23df7ba admin admin] PATH_INFO: `/v3/projects/624f5188238c41ee956fd9d2eec8c9c8` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:18.374705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ba7a8-6a9f-4f6a-a851-64e1c23df7ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=624f5188238c41ee956fd9d2eec8c9c8)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:18.381104 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ba7a8-6a9f-4f6a-a851-64e1c23df7ba admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:18.551647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-886ba7a8-6a9f-4f6a-a851-64e1c23df7ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '624f5188238c41ee956fd9d2eec8c9c8', 'request_id': 'req-886ba7a8-6a9f-4f6a-a851-64e1c23df7ba'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:18.561906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-886ba7a8-6a9f-4f6a-a851-64e1c23df7ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '624f5188238c41ee956fd9d2eec8c9c8', 'request_id': 'req-886ba7a8-6a9f-4f6a-a851-64e1c23df7ba'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:18.564809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:18:18.575040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-886ba7a8-6a9f-4f6a-a851-64e1c23df7ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '624f5188238c41ee956fd9d2eec8c9c8', 'request_id': 'req-886ba7a8-6a9f-4f6a-a851-64e1c23df7ba'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:18.671322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-886ba7a8-6a9f-4f6a-a851-64e1c23df7ba admin admin] The token cache is being invalidate because project 624f5188238c41ee956fd9d2eec8c9c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:18:18.671539 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-886ba7a8-6a9f-4f6a-a851-64e1c23df7ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:18.672178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-886ba7a8-6a9f-4f6a-a851-64e1c23df7ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:18:18.673873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1213/2425] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:18:18 2022] DELETE /identity/v3/projects/624f5188238c41ee956fd9d2eec8c9c8 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:18:22.663374 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1bc94-b433-4dc7-8ded-39c35191cbdd service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:22.665217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-18c1bc94-b433-4dc7-8ded-39c35191cbdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:22.677764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1bc94-b433-4dc7-8ded-39c35191cbdd service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:22.715402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:18:22.736215 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed1811-79ba-42d8-aa94-5ef2a2d017cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:22.736976 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed1811-79ba-42d8-aa94-5ef2a2d017cc service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:22.737217 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed1811-79ba-42d8-aa94-5ef2a2d017cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:22.737465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed1811-79ba-42d8-aa94-5ef2a2d017cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:22.739375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e3ed1811-79ba-42d8-aa94-5ef2a2d017cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:22.818490 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed1811-79ba-42d8-aa94-5ef2a2d017cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:22.820493 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed1811-79ba-42d8-aa94-5ef2a2d017cc service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:22.849722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1213/2426] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:18:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:18:28.899479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=4,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:18:28.979530 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf0a82-c653-4565-9f4c-ea4e9557ce1d None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:28.979834 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf0a82-c653-4565-9f4c-ea4e9557ce1d None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:28.980118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf0a82-c653-4565-9f4c-ea4e9557ce1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:29.001383 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b5bf0a82-c653-4565-9f4c-ea4e9557ce1d None None] MFA Rules not processed for user `f3f0c37e426640f9960c1da232605e55`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:18:29.002686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b5bf0a82-c653-4565-9f4c-ea4e9557ce1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:29.062259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1214/2427] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:18:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:18:29.109291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed1811-79ba-42d8-aa94-5ef2a2d017cc service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:29.111360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e3ed1811-79ba-42d8-aa94-5ef2a2d017cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:29.196478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed1811-79ba-42d8-aa94-5ef2a2d017cc service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:29.201806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da33a6-8883-4823-bc2c-ed74776b9246 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8dcb0a0f2c4a2d955544324213739c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:29.202479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da33a6-8883-4823-bc2c-ed74776b9246 service glance] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:29.202896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da33a6-8883-4823-bc2c-ed74776b9246 service glance] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:29.203438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da33a6-8883-4823-bc2c-ed74776b9246 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:29.206120 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-28da33a6-8883-4823-bc2c-ed74776b9246 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:29.225728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da33a6-8883-4823-bc2c-ed74776b9246 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:29.228047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da33a6-8883-4823-bc2c-ed74776b9246 service glance] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:29.240140 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1214/2428] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:18:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:18:37.950738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf0a82-c653-4565-9f4c-ea4e9557ce1d None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:37.952317 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b5bf0a82-c653-4565-9f4c-ea4e9557ce1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:37.969132 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf0a82-c653-4565-9f4c-ea4e9557ce1d None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:37.974718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9ed67-17b7-4fab-a42d-89919220bff9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:37.975391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9ed67-17b7-4fab-a42d-89919220bff9 service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:37.975677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9ed67-17b7-4fab-a42d-89919220bff9 service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:37.976457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9ed67-17b7-4fab-a42d-89919220bff9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:37.978389 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-fbe9ed67-17b7-4fab-a42d-89919220bff9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:38.062322 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9ed67-17b7-4fab-a42d-89919220bff9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:38.064003 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9ed67-17b7-4fab-a42d-89919220bff9 service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:38.075346 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1215/2429] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:18:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:18:39.235629 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=64 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:18:42.584975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da33a6-8883-4823-bc2c-ed74776b9246 service glance] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:42.597494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da33a6-8883-4823-bc2c-ed74776b9246 service glance] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:42.607003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdead508-48b1-4f25-9984-e5930fa15711 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:42.607867 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdead508-48b1-4f25-9984-e5930fa15711 service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:42.608254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdead508-48b1-4f25-9984-e5930fa15711 service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:42.608641 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdead508-48b1-4f25-9984-e5930fa15711 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:42.610945 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-cdead508-48b1-4f25-9984-e5930fa15711 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:42.624716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdead508-48b1-4f25-9984-e5930fa15711 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:42.626519 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdead508-48b1-4f25-9984-e5930fa15711 service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:42.640505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1215/2430] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:18:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:18:48.070318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=53 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:18:50.556175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9ed67-17b7-4fab-a42d-89919220bff9 service nova] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:50.558801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-fbe9ed67-17b7-4fab-a42d-89919220bff9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:50.574342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9ed67-17b7-4fab-a42d-89919220bff9 service nova] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:50.578576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a7594-6090-4985-8d64-7af732446ee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:50.579247 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a7594-6090-4985-8d64-7af732446ee1 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:50.579538 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a7594-6090-4985-8d64-7af732446ee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:50.579816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a7594-6090-4985-8d64-7af732446ee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:50.581995 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9b1a7594-6090-4985-8d64-7af732446ee1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:50.668930 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a7594-6090-4985-8d64-7af732446ee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:50.670725 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a7594-6090-4985-8d64-7af732446ee1 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:50.693460 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1216/2431] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:18:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:18:52.634894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=6 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:18:53.359945 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033c0eb-7fb1-4134-9a17-8927515e5363 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:53.361085 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033c0eb-7fb1-4134-9a17-8927515e5363 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:53.361813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033c0eb-7fb1-4134-9a17-8927515e5363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:53.402858 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-3033c0eb-7fb1-4134-9a17-8927515e5363 None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:18:53.402858 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3033c0eb-7fb1-4134-9a17-8927515e5363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:53.494726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1216/2432] 10.222.0.83 () {58 vars in 967 bytes} [Mon Jul 18 15:18:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:18:53.525011 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a7594-6090-4985-8d64-7af732446ee1 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:53.530606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9b1a7594-6090-4985-8d64-7af732446ee1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:53.659454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a7594-6090-4985-8d64-7af732446ee1 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:53.670631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b6f4b-0383-4a76-8403-1079e5630f6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:53.671528 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b6f4b-0383-4a76-8403-1079e5630f6d service swift] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:53.674147 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b6f4b-0383-4a76-8403-1079e5630f6d service swift] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:53.674836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b6f4b-0383-4a76-8403-1079e5630f6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:53.691727 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b6f4b-0383-4a76-8403-1079e5630f6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:53.695722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b6f4b-0383-4a76-8403-1079e5630f6d service swift] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:53.708767 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1217/2433] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:18:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 15:18:56.777142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a5722-cfdf-41db-a7ac-13a0abefcaac None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:56.777725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a5722-cfdf-41db-a7ac-13a0abefcaac None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:56.777725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a5722-cfdf-41db-a7ac-13a0abefcaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:56.797376 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-9f6a5722-cfdf-41db-a7ac-13a0abefcaac None None] MFA Rules not processed for user `22b86eef7820462fa4e6cbb22f30b853`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:18:56.798679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-9f6a5722-cfdf-41db-a7ac-13a0abefcaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:56.806608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1217/2434] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:18:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:18:57.204120 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b6f4b-0383-4a76-8403-1079e5630f6d service swift] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:57.206423 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-444b6f4b-0383-4a76-8403-1079e5630f6d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:57.222171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b6f4b-0383-4a76-8403-1079e5630f6d service swift] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:57.227308 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8863dfc-e60c-42bf-8fe2-6772f5a6fa46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:57.227985 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8863dfc-e60c-42bf-8fe2-6772f5a6fa46 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:57.228274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8863dfc-e60c-42bf-8fe2-6772f5a6fa46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:57.228556 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8863dfc-e60c-42bf-8fe2-6772f5a6fa46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:57.231286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c8863dfc-e60c-42bf-8fe2-6772f5a6fa46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:57.244504 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8863dfc-e60c-42bf-8fe2-6772f5a6fa46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:57.246447 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8863dfc-e60c-42bf-8fe2-6772f5a6fa46 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:57.256559 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1218/2435] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:18:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:18:57.559183 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a5722-cfdf-41db-a7ac-13a0abefcaac None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:57.571853 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a5722-cfdf-41db-a7ac-13a0abefcaac None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:57.589965 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a415f00-40b0-4ceb-a489-db71713cd34a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:57.590682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a415f00-40b0-4ceb-a489-db71713cd34a service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:57.590977 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a415f00-40b0-4ceb-a489-db71713cd34a service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:57.591261 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a415f00-40b0-4ceb-a489-db71713cd34a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:57.593310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3a415f00-40b0-4ceb-a489-db71713cd34a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:57.619726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a415f00-40b0-4ceb-a489-db71713cd34a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:57.621249 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a415f00-40b0-4ceb-a489-db71713cd34a service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:57.639384 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1218/2436] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:18:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:18:57.808061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8863dfc-e60c-42bf-8fe2-6772f5a6fa46 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:18:57.811968 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c8863dfc-e60c-42bf-8fe2-6772f5a6fa46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:57.940340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8863dfc-e60c-42bf-8fe2-6772f5a6fa46 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:18:57.948558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448b69e-fdf3-42b9-8eb9-f20cc4ea9de9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76cc0ae48b5a49c6a7a1c11a08c389e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:18:57.949352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448b69e-fdf3-42b9-8eb9-f20cc4ea9de9 service placement] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:18:57.949713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448b69e-fdf3-42b9-8eb9-f20cc4ea9de9 service placement] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:18:57.950082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448b69e-fdf3-42b9-8eb9-f20cc4ea9de9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:18:57.952109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-1448b69e-fdf3-42b9-8eb9-f20cc4ea9de9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:18:57.965838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448b69e-fdf3-42b9-8eb9-f20cc4ea9de9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:18:57.967715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448b69e-fdf3-42b9-8eb9-f20cc4ea9de9 service placement] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:18:57.976401 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:18:57.984281 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1219/2437] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:18:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:19:02.120543 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a415f00-40b0-4ceb-a489-db71713cd34a service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:02.123530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3a415f00-40b0-4ceb-a489-db71713cd34a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:02.140918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a415f00-40b0-4ceb-a489-db71713cd34a service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:02.146749 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b79327-18f9-4eaf-aa12-448899a936d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76cc0ae48b5a49c6a7a1c11a08c389e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:02.149290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b79327-18f9-4eaf-aa12-448899a936d7 service placement] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:02.149290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b79327-18f9-4eaf-aa12-448899a936d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:02.149290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b79327-18f9-4eaf-aa12-448899a936d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:02.151473 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a9b79327-18f9-4eaf-aa12-448899a936d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:02.164783 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b79327-18f9-4eaf-aa12-448899a936d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:02.166429 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b79327-18f9-4eaf-aa12-448899a936d7 service placement] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:02.181476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1219/2438] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:19:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:19:05.968430 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448b69e-fdf3-42b9-8eb9-f20cc4ea9de9 service placement] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:05.986416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448b69e-fdf3-42b9-8eb9-f20cc4ea9de9 service placement] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:05.996598 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7d5b7-d6d5-45e1-ada0-ee4ff4b09772 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:05.997311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7d5b7-d6d5-45e1-ada0-ee4ff4b09772 service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:05.997666 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7d5b7-d6d5-45e1-ada0-ee4ff4b09772 service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:05.997946 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7d5b7-d6d5-45e1-ada0-ee4ff4b09772 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:06.000073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-90e7d5b7-d6d5-45e1-ada0-ee4ff4b09772 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:06.016331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7d5b7-d6d5-45e1-ada0-ee4ff4b09772 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:06.017974 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7d5b7-d6d5-45e1-ada0-ee4ff4b09772 service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:06.031334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1220/2439] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:19:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:19:06.997327 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b79327-18f9-4eaf-aa12-448899a936d7 service placement] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:06.999280 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a9b79327-18f9-4eaf-aa12-448899a936d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:07.061434 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b79327-18f9-4eaf-aa12-448899a936d7 service placement] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:07.065411 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5339d1-af59-49d1-b67e-17f798f2db3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:07.066078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5339d1-af59-49d1-b67e-17f798f2db3f service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:07.066370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5339d1-af59-49d1-b67e-17f798f2db3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:07.066688 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5339d1-af59-49d1-b67e-17f798f2db3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:07.068616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7a5339d1-af59-49d1-b67e-17f798f2db3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:07.168501 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5339d1-af59-49d1-b67e-17f798f2db3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:07.170061 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5339d1-af59-49d1-b67e-17f798f2db3f service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:07.184843 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1220/2440] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:19:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:19:07.840937 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7d5b7-d6d5-45e1-ada0-ee4ff4b09772 service nova] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:07.843871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-90e7d5b7-d6d5-45e1-ada0-ee4ff4b09772 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:07.949665 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7d5b7-d6d5-45e1-ada0-ee4ff4b09772 service nova] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:07.958045 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b720e0d-9981-4c4a-874c-62ff6fa3bc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:07.960513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b720e0d-9981-4c4a-874c-62ff6fa3bc1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:07.961027 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b720e0d-9981-4c4a-874c-62ff6fa3bc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:07.961581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b720e0d-9981-4c4a-874c-62ff6fa3bc1d admin admin] PATH_INFO: `/v3/users/45b1df340f8a4102adf5f680d7e64139` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:07.963758 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b720e0d-9981-4c4a-874c-62ff6fa3bc1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=45b1df340f8a4102adf5f680d7e64139)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:07.965942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b720e0d-9981-4c4a-874c-62ff6fa3bc1d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:08.214609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-2b720e0d-9981-4c4a-874c-62ff6fa3bc1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45b1df340f8a4102adf5f680d7e64139'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:08.215227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-2b720e0d-9981-4c4a-874c-62ff6fa3bc1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45b1df340f8a4102adf5f680d7e64139'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:08.216116 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-2b720e0d-9981-4c4a-874c-62ff6fa3bc1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45b1df340f8a4102adf5f680d7e64139'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:08.294955 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-2b720e0d-9981-4c4a-874c-62ff6fa3bc1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45b1df340f8a4102adf5f680d7e64139'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:08.321721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-2b720e0d-9981-4c4a-874c-62ff6fa3bc1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45b1df340f8a4102adf5f680d7e64139'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:08.340736 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1221/2441] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:19:07 2022] DELETE /identity/v3/users/45b1df340f8a4102adf5f680d7e64139 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:08.662563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5339d1-af59-49d1-b67e-17f798f2db3f service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:08.681790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5339d1-af59-49d1-b67e-17f798f2db3f service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:08.705524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:08.752586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584a3fa-48d0-4461-8eaa-08121738a8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:08.753460 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584a3fa-48d0-4461-8eaa-08121738a8f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:08.758011 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584a3fa-48d0-4461-8eaa-08121738a8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:08.758011 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584a3fa-48d0-4461-8eaa-08121738a8f7 admin admin] PATH_INFO: `/v3/projects/eb8e493ee52b4c2686d5010eed312191` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:08.758011 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584a3fa-48d0-4461-8eaa-08121738a8f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb8e493ee52b4c2686d5010eed312191)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:08.758011 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584a3fa-48d0-4461-8eaa-08121738a8f7 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:08.881726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e584a3fa-48d0-4461-8eaa-08121738a8f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb8e493ee52b4c2686d5010eed312191', 'request_id': 'req-e584a3fa-48d0-4461-8eaa-08121738a8f7'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:08.889119 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e584a3fa-48d0-4461-8eaa-08121738a8f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb8e493ee52b4c2686d5010eed312191', 'request_id': 'req-e584a3fa-48d0-4461-8eaa-08121738a8f7'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:08.974750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e584a3fa-48d0-4461-8eaa-08121738a8f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb8e493ee52b4c2686d5010eed312191', 'request_id': 'req-e584a3fa-48d0-4461-8eaa-08121738a8f7'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:08.988661 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e584a3fa-48d0-4461-8eaa-08121738a8f7 admin admin] The token cache is being invalidate because project eb8e493ee52b4c2686d5010eed312191 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:19:08.988954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e584a3fa-48d0-4461-8eaa-08121738a8f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:08.989633 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e584a3fa-48d0-4461-8eaa-08121738a8f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:08.993193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1221/2442] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:19:08 2022] DELETE /identity/v3/projects/eb8e493ee52b4c2686d5010eed312191 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:08.999140 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b720e0d-9981-4c4a-874c-62ff6fa3bc1d admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:09.000608 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2b720e0d-9981-4c4a-874c-62ff6fa3bc1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:09.014574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b720e0d-9981-4c4a-874c-62ff6fa3bc1d admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:09.077989 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef0d51-c772-43cf-a748-1f2948efcabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:09.078914 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef0d51-c772-43cf-a748-1f2948efcabc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:09.079196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef0d51-c772-43cf-a748-1f2948efcabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:09.079472 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef0d51-c772-43cf-a748-1f2948efcabc admin admin] PATH_INFO: `/v3/users/ac71e333fa5d40b0b0b8d8fe77d3c68b` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:09.080689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef0d51-c772-43cf-a748-1f2948efcabc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac71e333fa5d40b0b0b8d8fe77d3c68b)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:09.082749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef0d51-c772-43cf-a748-1f2948efcabc admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:09.090385 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:09.395153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-baef0d51-c772-43cf-a748-1f2948efcabc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac71e333fa5d40b0b0b8d8fe77d3c68b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:09.395895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-baef0d51-c772-43cf-a748-1f2948efcabc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac71e333fa5d40b0b0b8d8fe77d3c68b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:09.396561 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-baef0d51-c772-43cf-a748-1f2948efcabc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac71e333fa5d40b0b0b8d8fe77d3c68b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:09.500454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-baef0d51-c772-43cf-a748-1f2948efcabc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac71e333fa5d40b0b0b8d8fe77d3c68b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:09.513897 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-baef0d51-c772-43cf-a748-1f2948efcabc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac71e333fa5d40b0b0b8d8fe77d3c68b'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:09.528569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1222/2443] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:19:08 2022] DELETE /identity/v3/users/ac71e333fa5d40b0b0b8d8fe77d3c68b => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:19:09.770579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584a3fa-48d0-4461-8eaa-08121738a8f7 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:09.772088 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e584a3fa-48d0-4461-8eaa-08121738a8f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:09.788776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584a3fa-48d0-4461-8eaa-08121738a8f7 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:09.857290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:09.858168 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:09.858459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:09.858774 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c admin admin] PATH_INFO: `/v3/projects/608fd13d890e4744b9c6213232b47433` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:09.860023 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=608fd13d890e4744b9c6213232b47433)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:09.862004 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:10.011526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '608fd13d890e4744b9c6213232b47433', 'request_id': 'req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:10.019050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '608fd13d890e4744b9c6213232b47433', 'request_id': 'req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:10.023250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:10.134525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '608fd13d890e4744b9c6213232b47433', 'request_id': 'req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:10.153663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c admin admin] The token cache is being invalidate because project 608fd13d890e4744b9c6213232b47433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:19:10.153948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:10.154692 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:10.156389 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1222/2444] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:19:09 2022] DELETE /identity/v3/projects/608fd13d890e4744b9c6213232b47433 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:12.220733 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ec158-6efa-4638-882b-85e1163a6c4e None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:12.221349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ec158-6efa-4638-882b-85e1163a6c4e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:12.221349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ec158-6efa-4638-882b-85e1163a6c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:12.239929 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-c93ec158-6efa-4638-882b-85e1163a6c4e None None] MFA Rules not processed for user `4746f061cfd148609ffaf568f869d6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:19:12.241290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c93ec158-6efa-4638-882b-85e1163a6c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:12.306145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1223/2445] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:19:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:19:20.208562 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=4,keystone:DELETE=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:22.298264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=48,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:22.383417 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:22.384402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:22.493983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342ed4a-1f7e-45d0-9e40-ff07f5d81b8c admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:22.500818 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790386b5-9b12-4edc-871b-ea0f8bbafdd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:22.501490 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790386b5-9b12-4edc-871b-ea0f8bbafdd9 service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:22.501777 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790386b5-9b12-4edc-871b-ea0f8bbafdd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:22.502058 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790386b5-9b12-4edc-871b-ea0f8bbafdd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:22.503963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-790386b5-9b12-4edc-871b-ea0f8bbafdd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:22.517309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790386b5-9b12-4edc-871b-ea0f8bbafdd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:22.519003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790386b5-9b12-4edc-871b-ea0f8bbafdd9 service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:22.532278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1223/2446] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:19:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:19:29.593185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ec158-6efa-4638-882b-85e1163a6c4e None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:29.614107 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ec158-6efa-4638-882b-85e1163a6c4e None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:29.621832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05138311-566b-4cad-bb9e-1335c1537495 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:29.622508 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05138311-566b-4cad-bb9e-1335c1537495 service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:29.622838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05138311-566b-4cad-bb9e-1335c1537495 service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:29.623124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05138311-566b-4cad-bb9e-1335c1537495 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:29.625062 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-05138311-566b-4cad-bb9e-1335c1537495 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:29.638811 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05138311-566b-4cad-bb9e-1335c1537495 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:29.640454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05138311-566b-4cad-bb9e-1335c1537495 service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:29.652243 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1224/2447] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:19:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:19:32.526711 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=34 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:36.141702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790386b5-9b12-4edc-871b-ea0f8bbafdd9 service nova] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:36.143309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-790386b5-9b12-4edc-871b-ea0f8bbafdd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:36.204761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790386b5-9b12-4edc-871b-ea0f8bbafdd9 service nova] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:36.208435 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5404468-5800-4978-81dc-db6e42a02ff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:36.209177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5404468-5800-4978-81dc-db6e42a02ff7 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:36.209371 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5404468-5800-4978-81dc-db6e42a02ff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:36.209651 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5404468-5800-4978-81dc-db6e42a02ff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:36.211656 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b5404468-5800-4978-81dc-db6e42a02ff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:36.276850 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5404468-5800-4978-81dc-db6e42a02ff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:36.278327 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5404468-5800-4978-81dc-db6e42a02ff7 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:36.289859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1224/2448] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:19:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:19:39.647102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=6 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:40.030996 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05138311-566b-4cad-bb9e-1335c1537495 service nova] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:40.032819 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-05138311-566b-4cad-bb9e-1335c1537495 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:40.112755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05138311-566b-4cad-bb9e-1335c1537495 service nova] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:40.118722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b4924-db8b-4610-9bac-7881fe9c5ff6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76cc0ae48b5a49c6a7a1c11a08c389e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:40.121158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b4924-db8b-4610-9bac-7881fe9c5ff6 service placement] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:40.121158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b4924-db8b-4610-9bac-7881fe9c5ff6 service placement] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:40.121158 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b4924-db8b-4610-9bac-7881fe9c5ff6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:40.121761 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d52b4924-db8b-4610-9bac-7881fe9c5ff6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:40.134026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b4924-db8b-4610-9bac-7881fe9c5ff6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:40.135796 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b4924-db8b-4610-9bac-7881fe9c5ff6 service placement] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:40.151749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1225/2449] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:19:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:19:40.185221 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5404468-5800-4978-81dc-db6e42a02ff7 service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:40.186139 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b5404468-5800-4978-81dc-db6e42a02ff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:40.200005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5404468-5800-4978-81dc-db6e42a02ff7 service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:40.208070 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b44db-0c2a-44a8-b469-52721f5fcfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:40.208070 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b44db-0c2a-44a8-b469-52721f5fcfa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:40.208070 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b44db-0c2a-44a8-b469-52721f5fcfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:40.208070 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b44db-0c2a-44a8-b469-52721f5fcfa3 admin admin] PATH_INFO: `/v3/users/648c869550ee437a8e775e997c741e58` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:40.208781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b44db-0c2a-44a8-b469-52721f5fcfa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=648c869550ee437a8e775e997c741e58)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:40.209774 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b44db-0c2a-44a8-b469-52721f5fcfa3 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:40.431646 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4e0b44db-0c2a-44a8-b469-52721f5fcfa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '648c869550ee437a8e775e997c741e58'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:40.438782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4e0b44db-0c2a-44a8-b469-52721f5fcfa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '648c869550ee437a8e775e997c741e58'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:40.439522 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4e0b44db-0c2a-44a8-b469-52721f5fcfa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '648c869550ee437a8e775e997c741e58'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:40.452838 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4e0b44db-0c2a-44a8-b469-52721f5fcfa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '648c869550ee437a8e775e997c741e58'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:40.528690 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4e0b44db-0c2a-44a8-b469-52721f5fcfa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '648c869550ee437a8e775e997c741e58'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:40.535524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1225/2450] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:19:40 2022] DELETE /identity/v3/users/648c869550ee437a8e775e997c741e58 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:40.825369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b4924-db8b-4610-9bac-7881fe9c5ff6 service placement] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:40.838676 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b4924-db8b-4610-9bac-7881fe9c5ff6 service placement] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:40.902847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715ac85-51ac-4619-9220-a9c37cb97092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:40.903779 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715ac85-51ac-4619-9220-a9c37cb97092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:40.904086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715ac85-51ac-4619-9220-a9c37cb97092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:40.904368 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715ac85-51ac-4619-9220-a9c37cb97092 admin admin] PATH_INFO: `/v3/projects/0e83fff7c7524da9905cdb86ccdb565a` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:40.905594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715ac85-51ac-4619-9220-a9c37cb97092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e83fff7c7524da9905cdb86ccdb565a)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:40.907637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715ac85-51ac-4619-9220-a9c37cb97092 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:41.017513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c715ac85-51ac-4619-9220-a9c37cb97092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e83fff7c7524da9905cdb86ccdb565a', 'request_id': 'req-c715ac85-51ac-4619-9220-a9c37cb97092'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:41.024716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c715ac85-51ac-4619-9220-a9c37cb97092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e83fff7c7524da9905cdb86ccdb565a', 'request_id': 'req-c715ac85-51ac-4619-9220-a9c37cb97092'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:41.031616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c715ac85-51ac-4619-9220-a9c37cb97092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e83fff7c7524da9905cdb86ccdb565a', 'request_id': 'req-c715ac85-51ac-4619-9220-a9c37cb97092'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:41.083243 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c715ac85-51ac-4619-9220-a9c37cb97092 admin admin] The token cache is being invalidate because project 0e83fff7c7524da9905cdb86ccdb565a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:19:41.083555 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c715ac85-51ac-4619-9220-a9c37cb97092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:41.084181 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-c715ac85-51ac-4619-9220-a9c37cb97092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:41.085803 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1226/2451] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:19:40 2022] DELETE /identity/v3/projects/0e83fff7c7524da9905cdb86ccdb565a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:41.091510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20faf5-f8de-49bb-8757-3c4d629e568e None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:41.091797 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20faf5-f8de-49bb-8757-3c4d629e568e None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:41.092086 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20faf5-f8de-49bb-8757-3c4d629e568e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:41.111199 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-fd20faf5-f8de-49bb-8757-3c4d629e568e None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:19:41.112447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-fd20faf5-f8de-49bb-8757-3c4d629e568e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:41.131886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:41.177582 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1226/2452] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:19:41.186514 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f038179-15ba-4274-a51a-726f79857131 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:41.186839 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f038179-15ba-4274-a51a-726f79857131 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:41.187129 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f038179-15ba-4274-a51a-726f79857131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:41.207389 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-6f038179-15ba-4274-a51a-726f79857131 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:19:41.208812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6f038179-15ba-4274-a51a-726f79857131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:41.218613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1227/2453] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:19:41.224067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20faf5-f8de-49bb-8757-3c4d629e568e None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:41.234960 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20faf5-f8de-49bb-8757-3c4d629e568e None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:41.239786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e7ba6-29fe-4164-ab58-70552a1eada7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:41.240445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e7ba6-29fe-4164-ab58-70552a1eada7 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:41.240728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e7ba6-29fe-4164-ab58-70552a1eada7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:41.241002 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e7ba6-29fe-4164-ab58-70552a1eada7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:41.241821 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e7ba6-29fe-4164-ab58-70552a1eada7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:19:41.242659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e7ba6-29fe-4164-ab58-70552a1eada7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:41.244720 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e7ba6-29fe-4164-ab58-70552a1eada7 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:41.257133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1227/2454] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:19:41.262505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f038179-15ba-4274-a51a-726f79857131 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:41.272255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f038179-15ba-4274-a51a-726f79857131 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:41.277558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f357bb-bada-44f0-b017-968bf9b8ed7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:41.278236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f357bb-bada-44f0-b017-968bf9b8ed7f admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:41.278527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f357bb-bada-44f0-b017-968bf9b8ed7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:41.281056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f357bb-bada-44f0-b017-968bf9b8ed7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:41.281056 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f357bb-bada-44f0-b017-968bf9b8ed7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:41.281501 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f357bb-bada-44f0-b017-968bf9b8ed7f admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:41.292515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:41.365317 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1228/2455] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:19:41 2022] POST /identity/v3/projects => generated 401 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:19:41.370961 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e7ba6-29fe-4164-ab58-70552a1eada7 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:41.381504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e7ba6-29fe-4164-ab58-70552a1eada7 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:41.441031 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998b538-c846-4478-a8f8-5d503c2b4437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:41.441873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998b538-c846-4478-a8f8-5d503c2b4437 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:41.442168 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998b538-c846-4478-a8f8-5d503c2b4437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:41.442452 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998b538-c846-4478-a8f8-5d503c2b4437 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:41.443188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998b538-c846-4478-a8f8-5d503c2b4437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:41.445341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998b538-c846-4478-a8f8-5d503c2b4437 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:41.560038 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1228/2456] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:19:41 2022] POST /identity/v3/users => generated 301 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:19:41.564474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f357bb-bada-44f0-b017-968bf9b8ed7f admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:41.576007 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f357bb-bada-44f0-b017-968bf9b8ed7f admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:41.581328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3446a-d9ab-436e-ab02-f61dc8c15a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:41.582010 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3446a-d9ab-436e-ab02-f61dc8c15a29 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:41.582301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3446a-d9ab-436e-ab02-f61dc8c15a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:41.582620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3446a-d9ab-436e-ab02-f61dc8c15a29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:41.583355 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3446a-d9ab-436e-ab02-f61dc8c15a29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:41.585053 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3446a-d9ab-436e-ab02-f61dc8c15a29 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:41.596849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1229/2457] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:19:41.601112 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998b538-c846-4478-a8f8-5d503c2b4437 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:41.611927 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998b538-c846-4478-a8f8-5d503c2b4437 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:41.616742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f4997-f8b8-4e3d-ba5f-0077d4666d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:41.618275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f4997-f8b8-4e3d-ba5f-0077d4666d7b admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:41.618588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f4997-f8b8-4e3d-ba5f-0077d4666d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:41.618894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f4997-f8b8-4e3d-ba5f-0077d4666d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:41.619620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f4997-f8b8-4e3d-ba5f-0077d4666d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:41.621250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f4997-f8b8-4e3d-ba5f-0077d4666d7b admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:41.631751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1229/2458] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:19:41.636361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3446a-d9ab-436e-ab02-f61dc8c15a29 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:41.647274 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3446a-d9ab-436e-ab02-f61dc8c15a29 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:41.654332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79afce5-667a-4c47-8e1a-3270876bdabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:41.655267 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79afce5-667a-4c47-8e1a-3270876bdabc admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:41.655557 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79afce5-667a-4c47-8e1a-3270876bdabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:41.655838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79afce5-667a-4c47-8e1a-3270876bdabc admin admin] PATH_INFO: `/v3/projects/d9045695607846cc85bf08dcd3ebf65e/users/b319679319314472bae221d25f94d10d/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:41.672524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79afce5-667a-4c47-8e1a-3270876bdabc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9045695607846cc85bf08dcd3ebf65e, user_id=b319679319314472bae221d25f94d10d, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:41.675249 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79afce5-667a-4c47-8e1a-3270876bdabc admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:41.776227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1230/2459] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:19:41 2022] PUT /identity/v3/projects/d9045695607846cc85bf08dcd3ebf65e/users/b319679319314472bae221d25f94d10d/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:41.788450 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6f85b-094e-471d-b8f6-a945ed1b57c2 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:41.792803 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6f85b-094e-471d-b8f6-a945ed1b57c2 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:41.792803 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6f85b-094e-471d-b8f6-a945ed1b57c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:41.819764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-cde6f85b-094e-471d-b8f6-a945ed1b57c2 None None] MFA Rules not processed for user `b319679319314472bae221d25f94d10d`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:19:41.821358 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-cde6f85b-094e-471d-b8f6-a945ed1b57c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:41.908410 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1230/2460] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:19:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:19:41.915907 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79afce5-667a-4c47-8e1a-3270876bdabc admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:41.984325 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79afce5-667a-4c47-8e1a-3270876bdabc admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:41.989332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b3065-559c-4a57-94dc-f4d56002ff76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:41.989913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b3065-559c-4a57-94dc-f4d56002ff76 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:41.990197 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b3065-559c-4a57-94dc-f4d56002ff76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:41.990469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b3065-559c-4a57-94dc-f4d56002ff76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:41.991371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b3065-559c-4a57-94dc-f4d56002ff76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:41.994342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b3065-559c-4a57-94dc-f4d56002ff76 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:42.074668 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1231/2461] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:19:41 2022] POST /identity/v3/projects => generated 399 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:19:42.078866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6f85b-094e-471d-b8f6-a945ed1b57c2 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:42.090661 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6f85b-094e-471d-b8f6-a945ed1b57c2 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:42.105530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:42.152048 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac414d-4442-4373-aa32-d1137c5e0f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:42.152725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac414d-4442-4373-aa32-d1137c5e0f3e admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:42.153030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac414d-4442-4373-aa32-d1137c5e0f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:42.153287 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac414d-4442-4373-aa32-d1137c5e0f3e admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:42.153982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac414d-4442-4373-aa32-d1137c5e0f3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:42.156021 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac414d-4442-4373-aa32-d1137c5e0f3e admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:42.296688 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1231/2462] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:19:42 2022] POST /identity/v3/users => generated 306 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:19:42.301179 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b3065-559c-4a57-94dc-f4d56002ff76 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:42.312717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b3065-559c-4a57-94dc-f4d56002ff76 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:42.317834 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bfa7d-7b05-4a96-9786-2cbc3ef11c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:42.318641 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bfa7d-7b05-4a96-9786-2cbc3ef11c81 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:42.318935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bfa7d-7b05-4a96-9786-2cbc3ef11c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:42.319357 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bfa7d-7b05-4a96-9786-2cbc3ef11c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:42.320301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83bfa7d-7b05-4a96-9786-2cbc3ef11c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:42.322355 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83bfa7d-7b05-4a96-9786-2cbc3ef11c81 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:42.338637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1232/2463] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:19:42.342263 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac414d-4442-4373-aa32-d1137c5e0f3e admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:42.353407 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac414d-4442-4373-aa32-d1137c5e0f3e admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:42.358345 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c59c4-4f49-4be3-8f57-b9ef05e49ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:42.359286 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c59c4-4f49-4be3-8f57-b9ef05e49ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:42.359578 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c59c4-4f49-4be3-8f57-b9ef05e49ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:42.359859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c59c4-4f49-4be3-8f57-b9ef05e49ad2 admin admin] PATH_INFO: `/v3/projects/c29d244a702e4ebf9168897523329180/users/1a6fc72acbf74566b2967351d54854a4/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:42.376536 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c59c4-4f49-4be3-8f57-b9ef05e49ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29d244a702e4ebf9168897523329180, user_id=1a6fc72acbf74566b2967351d54854a4, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:42.379287 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c59c4-4f49-4be3-8f57-b9ef05e49ad2 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:42.496419 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1232/2464] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:19:42 2022] PUT /identity/v3/projects/c29d244a702e4ebf9168897523329180/users/1a6fc72acbf74566b2967351d54854a4/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:42.500699 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bfa7d-7b05-4a96-9786-2cbc3ef11c81 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:42.513364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bfa7d-7b05-4a96-9786-2cbc3ef11c81 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:42.580242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94baf4c-6d02-4a5c-83f9-284967199457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:42.581050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94baf4c-6d02-4a5c-83f9-284967199457 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:42.581210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94baf4c-6d02-4a5c-83f9-284967199457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:42.581492 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94baf4c-6d02-4a5c-83f9-284967199457 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:42.582253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94baf4c-6d02-4a5c-83f9-284967199457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:42.583908 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94baf4c-6d02-4a5c-83f9-284967199457 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:42.596542 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1233/2465] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:19:42.600026 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c59c4-4f49-4be3-8f57-b9ef05e49ad2 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:42.612460 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c59c4-4f49-4be3-8f57-b9ef05e49ad2 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:42.617769 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79647a-ec59-47fb-832b-a2adf019e152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:42.618750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79647a-ec59-47fb-832b-a2adf019e152 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:42.619034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79647a-ec59-47fb-832b-a2adf019e152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:42.619315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79647a-ec59-47fb-832b-a2adf019e152 admin admin] PATH_INFO: `/v3/projects/c29d244a702e4ebf9168897523329180/users/1a6fc72acbf74566b2967351d54854a4/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:42.621357 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79647a-ec59-47fb-832b-a2adf019e152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29d244a702e4ebf9168897523329180, user_id=1a6fc72acbf74566b2967351d54854a4, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:42.624099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79647a-ec59-47fb-832b-a2adf019e152 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:42.636053 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1233/2466] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:19:42 2022] PUT /identity/v3/projects/c29d244a702e4ebf9168897523329180/users/1a6fc72acbf74566b2967351d54854a4/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:42.647195 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147c7cb-f926-4c2e-89b9-684c03d68840 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:42.647615 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147c7cb-f926-4c2e-89b9-684c03d68840 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:42.647911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147c7cb-f926-4c2e-89b9-684c03d68840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:42.677497 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d147c7cb-f926-4c2e-89b9-684c03d68840 None None] MFA Rules not processed for user `1a6fc72acbf74566b2967351d54854a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:19:42.678827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d147c7cb-f926-4c2e-89b9-684c03d68840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:42.698257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:42.757100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1234/2467] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:19:42 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:19:42.769004 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79647a-ec59-47fb-832b-a2adf019e152 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:42.770915 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-5f79647a-ec59-47fb-832b-a2adf019e152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:42.838207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79647a-ec59-47fb-832b-a2adf019e152 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:42.842175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc4505-970c-49f9-abbb-4592ca2fc2db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:42.844582 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc4505-970c-49f9-abbb-4592ca2fc2db service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:42.844582 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc4505-970c-49f9-abbb-4592ca2fc2db service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:42.844582 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc4505-970c-49f9-abbb-4592ca2fc2db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:42.854441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defc4505-970c-49f9-abbb-4592ca2fc2db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:42.855893 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defc4505-970c-49f9-abbb-4592ca2fc2db service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:42.867137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1234/2468] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:19:42 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:19:43.241239 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147c7cb-f926-4c2e-89b9-684c03d68840 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:43.253742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147c7cb-f926-4c2e-89b9-684c03d68840 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:43.257868 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f4a82-613f-47ec-9b20-ff0aa27ebb81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:43.258527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f4a82-613f-47ec-9b20-ff0aa27ebb81 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:43.258846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f4a82-613f-47ec-9b20-ff0aa27ebb81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:43.259126 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f4a82-613f-47ec-9b20-ff0aa27ebb81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:43.320631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f4a82-613f-47ec-9b20-ff0aa27ebb81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:43.322337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f4a82-613f-47ec-9b20-ff0aa27ebb81 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:43.331752 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1235/2469] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:19:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:19:49.751509 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc4505-970c-49f9-abbb-4592ca2fc2db service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:49.838685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc4505-970c-49f9-abbb-4592ca2fc2db service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:49.843652 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47931c5-98e2-4565-ae0c-c6779c997ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:49.844566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47931c5-98e2-4565-ae0c-c6779c997ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:49.844850 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47931c5-98e2-4565-ae0c-c6779c997ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:49.845390 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47931c5-98e2-4565-ae0c-c6779c997ac6 admin admin] PATH_INFO: `/v3/users/b319679319314472bae221d25f94d10d` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:49.846669 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47931c5-98e2-4565-ae0c-c6779c997ac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b319679319314472bae221d25f94d10d)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:49.848700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47931c5-98e2-4565-ae0c-c6779c997ac6 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:49.864006 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:50.107847 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e47931c5-98e2-4565-ae0c-c6779c997ac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b319679319314472bae221d25f94d10d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:50.116668 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e47931c5-98e2-4565-ae0c-c6779c997ac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b319679319314472bae221d25f94d10d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:50.118242 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e47931c5-98e2-4565-ae0c-c6779c997ac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b319679319314472bae221d25f94d10d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:50.134882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e47931c5-98e2-4565-ae0c-c6779c997ac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b319679319314472bae221d25f94d10d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:50.220986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e47931c5-98e2-4565-ae0c-c6779c997ac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b319679319314472bae221d25f94d10d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:50.230142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1235/2470] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:19:49 2022] DELETE /identity/v3/users/b319679319314472bae221d25f94d10d => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:50.235664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083509e5-f0fb-4521-93d6-e94f3dd0101c None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:50.236298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083509e5-f0fb-4521-93d6-e94f3dd0101c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:50.236298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083509e5-f0fb-4521-93d6-e94f3dd0101c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:50.257317 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-083509e5-f0fb-4521-93d6-e94f3dd0101c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:19:50.259346 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-083509e5-f0fb-4521-93d6-e94f3dd0101c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:50.352276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1236/2471] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:19:50.364850 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47931c5-98e2-4565-ae0c-c6779c997ac6 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:50.366392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e47931c5-98e2-4565-ae0c-c6779c997ac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:50.434270 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47931c5-98e2-4565-ae0c-c6779c997ac6 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:50.438171 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06238cfd-e8ea-4f67-9c41-ea03b24586b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:50.438885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06238cfd-e8ea-4f67-9c41-ea03b24586b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:50.439170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06238cfd-e8ea-4f67-9c41-ea03b24586b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:50.439447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06238cfd-e8ea-4f67-9c41-ea03b24586b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:50.449858 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06238cfd-e8ea-4f67-9c41-ea03b24586b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:50.451396 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06238cfd-e8ea-4f67-9c41-ea03b24586b5 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:50.462461 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1236/2472] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:19:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:19:51.307721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083509e5-f0fb-4521-93d6-e94f3dd0101c None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:51.309331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-083509e5-f0fb-4521-93d6-e94f3dd0101c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:51.325385 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083509e5-f0fb-4521-93d6-e94f3dd0101c None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:51.331113 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00709981-7aad-4f5f-a744-96a898643877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:51.332069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00709981-7aad-4f5f-a744-96a898643877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:51.332361 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00709981-7aad-4f5f-a744-96a898643877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:51.332649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00709981-7aad-4f5f-a744-96a898643877 admin admin] PATH_INFO: `/v3/projects/d9045695607846cc85bf08dcd3ebf65e` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:51.333970 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00709981-7aad-4f5f-a744-96a898643877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9045695607846cc85bf08dcd3ebf65e)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:51.336751 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00709981-7aad-4f5f-a744-96a898643877 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:51.472287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:51.476896 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00709981-7aad-4f5f-a744-96a898643877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9045695607846cc85bf08dcd3ebf65e', 'request_id': 'req-00709981-7aad-4f5f-a744-96a898643877'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:51.484244 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00709981-7aad-4f5f-a744-96a898643877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9045695607846cc85bf08dcd3ebf65e', 'request_id': 'req-00709981-7aad-4f5f-a744-96a898643877'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:51.490530 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00709981-7aad-4f5f-a744-96a898643877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9045695607846cc85bf08dcd3ebf65e', 'request_id': 'req-00709981-7aad-4f5f-a744-96a898643877'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:51.603685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00709981-7aad-4f5f-a744-96a898643877 admin admin] The token cache is being invalidate because project d9045695607846cc85bf08dcd3ebf65e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:19:51.603933 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00709981-7aad-4f5f-a744-96a898643877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:51.604714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00709981-7aad-4f5f-a744-96a898643877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:51.606334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1237/2473] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:19:51 2022] DELETE /identity/v3/projects/d9045695607846cc85bf08dcd3ebf65e => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:51.613176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06238cfd-e8ea-4f67-9c41-ea03b24586b5 service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:51.629565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06238cfd-e8ea-4f67-9c41-ea03b24586b5 service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:51.698500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a5754-1d10-4397-b17e-248ea25ec511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:51.699436 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a5754-1d10-4397-b17e-248ea25ec511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:51.699722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a5754-1d10-4397-b17e-248ea25ec511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:51.700008 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a5754-1d10-4397-b17e-248ea25ec511 admin admin] PATH_INFO: `/v3/users/1a6fc72acbf74566b2967351d54854a4` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:51.701321 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a5754-1d10-4397-b17e-248ea25ec511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a6fc72acbf74566b2967351d54854a4)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:51.703393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a5754-1d10-4397-b17e-248ea25ec511 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:51.912155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:51.923135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6d6a5754-1d10-4397-b17e-248ea25ec511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a6fc72acbf74566b2967351d54854a4'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:51.937196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6d6a5754-1d10-4397-b17e-248ea25ec511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a6fc72acbf74566b2967351d54854a4'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:51.938257 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6d6a5754-1d10-4397-b17e-248ea25ec511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a6fc72acbf74566b2967351d54854a4'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:51.955936 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6d6a5754-1d10-4397-b17e-248ea25ec511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a6fc72acbf74566b2967351d54854a4'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:52.056173 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-6d6a5754-1d10-4397-b17e-248ea25ec511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a6fc72acbf74566b2967351d54854a4'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:52.065287 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1237/2474] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:19:51 2022] DELETE /identity/v3/users/1a6fc72acbf74566b2967351d54854a4 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:52.625572 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00709981-7aad-4f5f-a744-96a898643877 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:52.627736 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-00709981-7aad-4f5f-a744-96a898643877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:52.643287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00709981-7aad-4f5f-a744-96a898643877 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:52.721543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a211b2-2c9d-4198-9598-09042269d203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:52.722457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a211b2-2c9d-4198-9598-09042269d203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:52.722772 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a211b2-2c9d-4198-9598-09042269d203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:52.723040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a211b2-2c9d-4198-9598-09042269d203 admin admin] PATH_INFO: `/v3/projects/c29d244a702e4ebf9168897523329180` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:52.724601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a211b2-2c9d-4198-9598-09042269d203 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c29d244a702e4ebf9168897523329180)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:52.727578 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a211b2-2c9d-4198-9598-09042269d203 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:52.875025 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-27a211b2-2c9d-4198-9598-09042269d203 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c29d244a702e4ebf9168897523329180', 'request_id': 'req-27a211b2-2c9d-4198-9598-09042269d203'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:52.883219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-27a211b2-2c9d-4198-9598-09042269d203 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c29d244a702e4ebf9168897523329180', 'request_id': 'req-27a211b2-2c9d-4198-9598-09042269d203'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:52.890734 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-27a211b2-2c9d-4198-9598-09042269d203 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c29d244a702e4ebf9168897523329180', 'request_id': 'req-27a211b2-2c9d-4198-9598-09042269d203'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:52.964433 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-27a211b2-2c9d-4198-9598-09042269d203 admin admin] The token cache is being invalidate because project c29d244a702e4ebf9168897523329180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:19:52.964738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-27a211b2-2c9d-4198-9598-09042269d203 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:52.965658 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-27a211b2-2c9d-4198-9598-09042269d203 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:52.967573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1238/2475] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:19:52 2022] DELETE /identity/v3/projects/c29d244a702e4ebf9168897523329180 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:52.973279 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ed207-a441-4fab-abc9-f07bbe278d9d None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:52.973984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ed207-a441-4fab-abc9-f07bbe278d9d None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:52.973984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ed207-a441-4fab-abc9-f07bbe278d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:52.998130 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-b16ed207-a441-4fab-abc9-f07bbe278d9d None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:19:52.999568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b16ed207-a441-4fab-abc9-f07bbe278d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:53.068480 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1238/2476] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:19:53.077530 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3d45b-7207-4432-a10d-043b7517b9a9 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:53.077842 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3d45b-7207-4432-a10d-043b7517b9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:53.078128 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3d45b-7207-4432-a10d-043b7517b9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:53.100214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-a9b3d45b-7207-4432-a10d-043b7517b9a9 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:19:53.102315 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a9b3d45b-7207-4432-a10d-043b7517b9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:53.111150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1239/2477] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:19:53.115650 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ed207-a441-4fab-abc9-f07bbe278d9d None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:53.128336 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ed207-a441-4fab-abc9-f07bbe278d9d None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:53.136311 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a999ff6-5ec0-4fa0-bb3e-2a1148d60dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:53.137471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a999ff6-5ec0-4fa0-bb3e-2a1148d60dea admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:53.137990 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a999ff6-5ec0-4fa0-bb3e-2a1148d60dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:53.138460 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a999ff6-5ec0-4fa0-bb3e-2a1148d60dea admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:53.139560 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a999ff6-5ec0-4fa0-bb3e-2a1148d60dea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:19:53.140087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a999ff6-5ec0-4fa0-bb3e-2a1148d60dea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:53.142588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a999ff6-5ec0-4fa0-bb3e-2a1148d60dea admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:53.159017 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1239/2478] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:19:53.164311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3d45b-7207-4432-a10d-043b7517b9a9 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:53.174835 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3d45b-7207-4432-a10d-043b7517b9a9 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:53.180727 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac4a01-b32d-4935-9b95-adc1f4713c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:53.181548 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac4a01-b32d-4935-9b95-adc1f4713c81 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:53.181778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac4a01-b32d-4935-9b95-adc1f4713c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:53.182059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac4a01-b32d-4935-9b95-adc1f4713c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:53.182821 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac4a01-b32d-4935-9b95-adc1f4713c81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:53.184919 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac4a01-b32d-4935-9b95-adc1f4713c81 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:53.307926 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1240/2479] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:19:53 2022] POST /identity/v3/projects => generated 387 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:19:53.312308 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a999ff6-5ec0-4fa0-bb3e-2a1148d60dea admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:53.325767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a999ff6-5ec0-4fa0-bb3e-2a1148d60dea admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:53.433338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad603f2f-bcf2-41ea-9fb8-b3f8b6209faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:53.434033 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad603f2f-bcf2-41ea-9fb8-b3f8b6209faa admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:53.434358 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad603f2f-bcf2-41ea-9fb8-b3f8b6209faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:53.434704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad603f2f-bcf2-41ea-9fb8-b3f8b6209faa admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:53.435402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad603f2f-bcf2-41ea-9fb8-b3f8b6209faa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:53.437443 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad603f2f-bcf2-41ea-9fb8-b3f8b6209faa admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:53.525495 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1240/2480] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:19:53 2022] POST /identity/v3/users => generated 294 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:19:53.530689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac4a01-b32d-4935-9b95-adc1f4713c81 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:53.542471 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac4a01-b32d-4935-9b95-adc1f4713c81 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:53.548321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080ea95-e97f-4aee-b4f1-ef190ecad228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:53.549293 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080ea95-e97f-4aee-b4f1-ef190ecad228 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:53.549707 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080ea95-e97f-4aee-b4f1-ef190ecad228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:53.550112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080ea95-e97f-4aee-b4f1-ef190ecad228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:53.551140 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080ea95-e97f-4aee-b4f1-ef190ecad228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:53.553028 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080ea95-e97f-4aee-b4f1-ef190ecad228 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:53.565553 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1241/2481] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:19:53.573349 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad603f2f-bcf2-41ea-9fb8-b3f8b6209faa admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:53.585036 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad603f2f-bcf2-41ea-9fb8-b3f8b6209faa admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:53.592148 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059682a0-f382-44c0-a447-fea6607fde11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:53.593016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059682a0-f382-44c0-a447-fea6607fde11 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:53.593424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059682a0-f382-44c0-a447-fea6607fde11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:53.593811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059682a0-f382-44c0-a447-fea6607fde11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:53.594751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059682a0-f382-44c0-a447-fea6607fde11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:53.596937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059682a0-f382-44c0-a447-fea6607fde11 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:53.608903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1241/2482] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:19:53.613811 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080ea95-e97f-4aee-b4f1-ef190ecad228 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:53.625537 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080ea95-e97f-4aee-b4f1-ef190ecad228 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:53.631469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb8cb3-c8ef-4338-8cb3-9a212806ab08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:53.633080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb8cb3-c8ef-4338-8cb3-9a212806ab08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:53.633416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb8cb3-c8ef-4338-8cb3-9a212806ab08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:53.633748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb8cb3-c8ef-4338-8cb3-9a212806ab08 admin admin] PATH_INFO: `/v3/projects/1dc5b555078148bda699606e90339b8e/users/94f16bb66e614150afb8cd11785187c8/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:53.654774 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb8cb3-c8ef-4338-8cb3-9a212806ab08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc5b555078148bda699606e90339b8e, user_id=94f16bb66e614150afb8cd11785187c8, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:53.660352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb8cb3-c8ef-4338-8cb3-9a212806ab08 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:53.771641 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:53.781789 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1242/2483] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:19:53 2022] PUT /identity/v3/projects/1dc5b555078148bda699606e90339b8e/users/94f16bb66e614150afb8cd11785187c8/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:19:53.795158 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca9caf-6993-45a6-878b-71bb83d8978b None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:53.795543 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca9caf-6993-45a6-878b-71bb83d8978b None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:53.796021 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca9caf-6993-45a6-878b-71bb83d8978b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:53.827223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-87ca9caf-6993-45a6-878b-71bb83d8978b None None] MFA Rules not processed for user `94f16bb66e614150afb8cd11785187c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:19:53.828486 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-87ca9caf-6993-45a6-878b-71bb83d8978b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:53.837992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:53.899705 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1242/2484] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:19:53.904899 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb8cb3-c8ef-4338-8cb3-9a212806ab08 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:53.974832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb8cb3-c8ef-4338-8cb3-9a212806ab08 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:53.980995 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f60b6-b741-4d79-838f-c475fdcc1ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:53.981680 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f60b6-b741-4d79-838f-c475fdcc1ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:53.981970 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f60b6-b741-4d79-838f-c475fdcc1ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:53.982257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f60b6-b741-4d79-838f-c475fdcc1ef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:53.983032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f60b6-b741-4d79-838f-c475fdcc1ef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:53.985067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f60b6-b741-4d79-838f-c475fdcc1ef6 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:54.089148 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1243/2485] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:19:53 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:19:54.090581 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca9caf-6993-45a6-878b-71bb83d8978b None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:54.104909 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca9caf-6993-45a6-878b-71bb83d8978b None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:54.169442 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a97b7-6895-4bbd-8e1a-afd488fab0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:54.169971 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a97b7-6895-4bbd-8e1a-afd488fab0a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:54.170263 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a97b7-6895-4bbd-8e1a-afd488fab0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:54.170744 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a97b7-6895-4bbd-8e1a-afd488fab0a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:54.171585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a97b7-6895-4bbd-8e1a-afd488fab0a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:54.173617 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a97b7-6895-4bbd-8e1a-afd488fab0a4 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:54.310919 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1243/2486] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:19:54 2022] POST /identity/v3/users => generated 300 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:19:54.311141 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f60b6-b741-4d79-838f-c475fdcc1ef6 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:54.321813 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f60b6-b741-4d79-838f-c475fdcc1ef6 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:54.327368 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581578ad-4a95-45c6-baec-d972212f7db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:54.328081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581578ad-4a95-45c6-baec-d972212f7db3 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:54.328374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581578ad-4a95-45c6-baec-d972212f7db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:54.328662 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581578ad-4a95-45c6-baec-d972212f7db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:54.329431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581578ad-4a95-45c6-baec-d972212f7db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:54.331204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581578ad-4a95-45c6-baec-d972212f7db3 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:54.345978 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1244/2487] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:19:54.351009 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a97b7-6895-4bbd-8e1a-afd488fab0a4 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:54.363332 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a97b7-6895-4bbd-8e1a-afd488fab0a4 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:54.370278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da97838-ccf5-489c-a356-9145d553d963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:54.371296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da97838-ccf5-489c-a356-9145d553d963 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:54.371589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da97838-ccf5-489c-a356-9145d553d963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:54.371876 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da97838-ccf5-489c-a356-9145d553d963 admin admin] PATH_INFO: `/v3/projects/4351a3551dbe4645a65a883e712dec14/users/53d958ab3a57460a8d3617b7cc3b427e/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:54.395223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da97838-ccf5-489c-a356-9145d553d963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4351a3551dbe4645a65a883e712dec14, user_id=53d958ab3a57460a8d3617b7cc3b427e, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:54.399245 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da97838-ccf5-489c-a356-9145d553d963 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:54.502701 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1244/2488] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:19:54 2022] PUT /identity/v3/projects/4351a3551dbe4645a65a883e712dec14/users/53d958ab3a57460a8d3617b7cc3b427e/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:54.507284 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581578ad-4a95-45c6-baec-d972212f7db3 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:54.519530 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581578ad-4a95-45c6-baec-d972212f7db3 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:54.620776 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b05346-92db-47bd-aa90-28358d9ea06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:54.621484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b05346-92db-47bd-aa90-28358d9ea06c admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:54.621794 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b05346-92db-47bd-aa90-28358d9ea06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:54.622097 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b05346-92db-47bd-aa90-28358d9ea06c admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:54.622848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b05346-92db-47bd-aa90-28358d9ea06c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:54.624526 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b05346-92db-47bd-aa90-28358d9ea06c admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:54.637390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1245/2489] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:19:54.642097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da97838-ccf5-489c-a356-9145d553d963 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:54.654632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da97838-ccf5-489c-a356-9145d553d963 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:54.660264 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9f8d0-83cd-4ebb-b493-313d9b64cfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:54.661208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9f8d0-83cd-4ebb-b493-313d9b64cfbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:54.661498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9f8d0-83cd-4ebb-b493-313d9b64cfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:54.661785 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9f8d0-83cd-4ebb-b493-313d9b64cfbf admin admin] PATH_INFO: `/v3/projects/4351a3551dbe4645a65a883e712dec14/users/53d958ab3a57460a8d3617b7cc3b427e/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:54.664588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9f8d0-83cd-4ebb-b493-313d9b64cfbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4351a3551dbe4645a65a883e712dec14, user_id=53d958ab3a57460a8d3617b7cc3b427e, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:54.667515 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9f8d0-83cd-4ebb-b493-313d9b64cfbf admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:54.681564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1245/2490] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:19:54 2022] PUT /identity/v3/projects/4351a3551dbe4645a65a883e712dec14/users/53d958ab3a57460a8d3617b7cc3b427e/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:19:54.693503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e1406-ab78-4ba9-95ed-f86d5f95170b None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:54.695489 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e1406-ab78-4ba9-95ed-f86d5f95170b None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:54.695489 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e1406-ab78-4ba9-95ed-f86d5f95170b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:54.735595 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-bb6e1406-ab78-4ba9-95ed-f86d5f95170b None None] MFA Rules not processed for user `53d958ab3a57460a8d3617b7cc3b427e`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:19:54.737077 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-bb6e1406-ab78-4ba9-95ed-f86d5f95170b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:54.815753 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:54.827312 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1246/2491] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:19:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:19:54.844675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9f8d0-83cd-4ebb-b493-313d9b64cfbf admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:54.847112 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-36e9f8d0-83cd-4ebb-b493-313d9b64cfbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:54.914743 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9f8d0-83cd-4ebb-b493-313d9b64cfbf admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:54.918818 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85952aa9-7568-434c-bb3c-6dff05f3ce28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:54.919489 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85952aa9-7568-434c-bb3c-6dff05f3ce28 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:54.919771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85952aa9-7568-434c-bb3c-6dff05f3ce28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:54.920068 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85952aa9-7568-434c-bb3c-6dff05f3ce28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:54.933042 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85952aa9-7568-434c-bb3c-6dff05f3ce28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:54.939905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85952aa9-7568-434c-bb3c-6dff05f3ce28 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:54.953330 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1246/2492] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:19:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:19:56.459655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e1406-ab78-4ba9-95ed-f86d5f95170b None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:56.533291 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e1406-ab78-4ba9-95ed-f86d5f95170b None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:56.537901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31764f84-f526-445b-9938-7e77ea416869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:56.538918 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31764f84-f526-445b-9938-7e77ea416869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:56.539324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31764f84-f526-445b-9938-7e77ea416869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:56.539611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31764f84-f526-445b-9938-7e77ea416869 admin admin] PATH_INFO: `/v3/users/94f16bb66e614150afb8cd11785187c8` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:56.540901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31764f84-f526-445b-9938-7e77ea416869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94f16bb66e614150afb8cd11785187c8)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:56.542893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31764f84-f526-445b-9938-7e77ea416869 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:56.763549 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-31764f84-f526-445b-9938-7e77ea416869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94f16bb66e614150afb8cd11785187c8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:56.764248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-31764f84-f526-445b-9938-7e77ea416869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94f16bb66e614150afb8cd11785187c8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:56.764856 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-31764f84-f526-445b-9938-7e77ea416869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94f16bb66e614150afb8cd11785187c8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:56.854925 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-31764f84-f526-445b-9938-7e77ea416869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94f16bb66e614150afb8cd11785187c8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:56.869491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-31764f84-f526-445b-9938-7e77ea416869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94f16bb66e614150afb8cd11785187c8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:56.882386 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1247/2493] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:19:56 2022] DELETE /identity/v3/users/94f16bb66e614150afb8cd11785187c8 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:56.887886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1e25f-62de-427b-abad-c30fdc861487 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:56.888414 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1e25f-62de-427b-abad-c30fdc861487 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:56.888514 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1e25f-62de-427b-abad-c30fdc861487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:56.895781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:56.908431 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-fdf1e25f-62de-427b-abad-c30fdc861487 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:19:56.909933 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-fdf1e25f-62de-427b-abad-c30fdc861487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:57.018712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1247/2494] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:19:57.031612 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31764f84-f526-445b-9938-7e77ea416869 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:57.033231 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-31764f84-f526-445b-9938-7e77ea416869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:57.125718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31764f84-f526-445b-9938-7e77ea416869 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:57.130178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9cd5c-b353-48c1-86aa-7ff876036210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:57.132227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9cd5c-b353-48c1-86aa-7ff876036210 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:57.132227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9cd5c-b353-48c1-86aa-7ff876036210 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:57.132227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9cd5c-b353-48c1-86aa-7ff876036210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:57.143570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9cd5c-b353-48c1-86aa-7ff876036210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:57.145124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9cd5c-b353-48c1-86aa-7ff876036210 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:57.158203 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1248/2495] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:19:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:19:57.657729 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1e25f-62de-427b-abad-c30fdc861487 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:57.659360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-fdf1e25f-62de-427b-abad-c30fdc861487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:57.678861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1e25f-62de-427b-abad-c30fdc861487 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:57.686089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497709e-4784-4c20-b848-ceec1c8cb7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:57.687254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497709e-4784-4c20-b848-ceec1c8cb7d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:57.687667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497709e-4784-4c20-b848-ceec1c8cb7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:57.688037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497709e-4784-4c20-b848-ceec1c8cb7d0 admin admin] PATH_INFO: `/v3/projects/1dc5b555078148bda699606e90339b8e` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:57.689951 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497709e-4784-4c20-b848-ceec1c8cb7d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dc5b555078148bda699606e90339b8e)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:57.692790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497709e-4784-4c20-b848-ceec1c8cb7d0 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:57.842490 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1497709e-4784-4c20-b848-ceec1c8cb7d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dc5b555078148bda699606e90339b8e', 'request_id': 'req-1497709e-4784-4c20-b848-ceec1c8cb7d0'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:57.855624 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1497709e-4784-4c20-b848-ceec1c8cb7d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dc5b555078148bda699606e90339b8e', 'request_id': 'req-1497709e-4784-4c20-b848-ceec1c8cb7d0'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:57.917800 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1497709e-4784-4c20-b848-ceec1c8cb7d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dc5b555078148bda699606e90339b8e', 'request_id': 'req-1497709e-4784-4c20-b848-ceec1c8cb7d0'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:57.932397 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1497709e-4784-4c20-b848-ceec1c8cb7d0 admin admin] The token cache is being invalidate because project 1dc5b555078148bda699606e90339b8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:19:57.932697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1497709e-4784-4c20-b848-ceec1c8cb7d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:57.933428 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-1497709e-4784-4c20-b848-ceec1c8cb7d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:57.935123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1248/2496] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:19:57 2022] DELETE /identity/v3/projects/1dc5b555078148bda699606e90339b8e => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:57.939411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9cd5c-b353-48c1-86aa-7ff876036210 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:57.950209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9cd5c-b353-48c1-86aa-7ff876036210 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:57.964820 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:58.024938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf53b2-72d6-4bd2-bdea-1f20fa3d55f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:58.025862 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf53b2-72d6-4bd2-bdea-1f20fa3d55f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:58.026152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf53b2-72d6-4bd2-bdea-1f20fa3d55f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:58.026436 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf53b2-72d6-4bd2-bdea-1f20fa3d55f7 admin admin] PATH_INFO: `/v3/users/53d958ab3a57460a8d3617b7cc3b427e` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:58.027837 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf53b2-72d6-4bd2-bdea-1f20fa3d55f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d958ab3a57460a8d3617b7cc3b427e)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:58.029859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf53b2-72d6-4bd2-bdea-1f20fa3d55f7 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:58.187988 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-d5bf53b2-72d6-4bd2-bdea-1f20fa3d55f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d958ab3a57460a8d3617b7cc3b427e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:58.188909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-d5bf53b2-72d6-4bd2-bdea-1f20fa3d55f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d958ab3a57460a8d3617b7cc3b427e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:58.189689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-d5bf53b2-72d6-4bd2-bdea-1f20fa3d55f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d958ab3a57460a8d3617b7cc3b427e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:58.282107 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-d5bf53b2-72d6-4bd2-bdea-1f20fa3d55f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d958ab3a57460a8d3617b7cc3b427e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:58.302495 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-d5bf53b2-72d6-4bd2-bdea-1f20fa3d55f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d958ab3a57460a8d3617b7cc3b427e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:58.318944 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1249/2497] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:19:57 2022] DELETE /identity/v3/users/53d958ab3a57460a8d3617b7cc3b427e => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:58.849883 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497709e-4784-4c20-b848-ceec1c8cb7d0 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:58.851939 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1497709e-4784-4c20-b848-ceec1c8cb7d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:58.865584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497709e-4784-4c20-b848-ceec1c8cb7d0 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:58.945613 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe7150-cfdf-4ab6-9c1a-8d51de411519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:58.946566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe7150-cfdf-4ab6-9c1a-8d51de411519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:58.946876 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe7150-cfdf-4ab6-9c1a-8d51de411519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:58.947188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe7150-cfdf-4ab6-9c1a-8d51de411519 admin admin] PATH_INFO: `/v3/projects/4351a3551dbe4645a65a883e712dec14` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:58.948436 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe7150-cfdf-4ab6-9c1a-8d51de411519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4351a3551dbe4645a65a883e712dec14)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:58.950680 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe7150-cfdf-4ab6-9c1a-8d51de411519 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:59.049162 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:59.064451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0afe7150-cfdf-4ab6-9c1a-8d51de411519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4351a3551dbe4645a65a883e712dec14', 'request_id': 'req-0afe7150-cfdf-4ab6-9c1a-8d51de411519'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:59.071645 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0afe7150-cfdf-4ab6-9c1a-8d51de411519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4351a3551dbe4645a65a883e712dec14', 'request_id': 'req-0afe7150-cfdf-4ab6-9c1a-8d51de411519'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:59.192468 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0afe7150-cfdf-4ab6-9c1a-8d51de411519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4351a3551dbe4645a65a883e712dec14', 'request_id': 'req-0afe7150-cfdf-4ab6-9c1a-8d51de411519'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:59.205296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0afe7150-cfdf-4ab6-9c1a-8d51de411519 admin admin] The token cache is being invalidate because project 4351a3551dbe4645a65a883e712dec14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:19:59.205586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0afe7150-cfdf-4ab6-9c1a-8d51de411519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:59.206219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0afe7150-cfdf-4ab6-9c1a-8d51de411519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:19:59.207815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1249/2498] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:19:58 2022] DELETE /identity/v3/projects/4351a3551dbe4645a65a883e712dec14 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:19:59.213257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe9849-1727-418c-bc5d-eb27d705ec19 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:59.213859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe9849-1727-418c-bc5d-eb27d705ec19 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:59.213859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe9849-1727-418c-bc5d-eb27d705ec19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:59.233603 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-5afe9849-1727-418c-bc5d-eb27d705ec19 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:19:59.234938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-5afe9849-1727-418c-bc5d-eb27d705ec19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:59.300659 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1250/2499] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:19:59.310896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3c01f-87ea-4946-8850-b88be5f32f25 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:59.311192 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3c01f-87ea-4946-8850-b88be5f32f25 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:59.311465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3c01f-87ea-4946-8850-b88be5f32f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:59.337889 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-f5d3c01f-87ea-4946-8850-b88be5f32f25 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:19:59.339265 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f5d3c01f-87ea-4946-8850-b88be5f32f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:19:59.348693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1250/2500] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:19:59.368412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe9849-1727-418c-bc5d-eb27d705ec19 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:59.386269 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe9849-1727-418c-bc5d-eb27d705ec19 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:59.392734 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fd40b-6b84-42cb-98ab-d4d6638faaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:59.393428 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fd40b-6b84-42cb-98ab-d4d6638faaf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:59.393724 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fd40b-6b84-42cb-98ab-d4d6638faaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:59.394009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fd40b-6b84-42cb-98ab-d4d6638faaf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:59.394835 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fd40b-6b84-42cb-98ab-d4d6638faaf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:19:59.395161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fd40b-6b84-42cb-98ab-d4d6638faaf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:59.396958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fd40b-6b84-42cb-98ab-d4d6638faaf7 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:59.411097 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1251/2501] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:19:59.418372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3c01f-87ea-4946-8850-b88be5f32f25 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:59.434449 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3c01f-87ea-4946-8850-b88be5f32f25 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:59.440695 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a54fe-b6e9-45cd-b0f6-44da5b18cad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:59.441512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a54fe-b6e9-45cd-b0f6-44da5b18cad3 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:59.441909 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a54fe-b6e9-45cd-b0f6-44da5b18cad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:59.442392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a54fe-b6e9-45cd-b0f6-44da5b18cad3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:59.443591 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a54fe-b6e9-45cd-b0f6-44da5b18cad3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:59.445823 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a54fe-b6e9-45cd-b0f6-44da5b18cad3 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:59.560819 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1251/2502] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:19:59 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:19:59.562582 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fd40b-6b84-42cb-98ab-d4d6638faaf7 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:59.594209 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fd40b-6b84-42cb-98ab-d4d6638faaf7 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:59.642656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:19:59.685626 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01600f55-8b69-4f6b-8e36-bc08e0c48cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:59.686325 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01600f55-8b69-4f6b-8e36-bc08e0c48cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:59.686650 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01600f55-8b69-4f6b-8e36-bc08e0c48cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:59.686937 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01600f55-8b69-4f6b-8e36-bc08e0c48cf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:59.687661 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01600f55-8b69-4f6b-8e36-bc08e0c48cf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:59.689681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01600f55-8b69-4f6b-8e36-bc08e0c48cf7 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:59.844345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1252/2503] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:19:59 2022] POST /identity/v3/users => generated 290 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:19:59.849267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a54fe-b6e9-45cd-b0f6-44da5b18cad3 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:59.860879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a54fe-b6e9-45cd-b0f6-44da5b18cad3 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:59.865951 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6022378-caf5-43c4-b6e4-783528280fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:59.867201 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6022378-caf5-43c4-b6e4-783528280fca admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:59.867492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6022378-caf5-43c4-b6e4-783528280fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:59.867778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6022378-caf5-43c4-b6e4-783528280fca admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:59.873639 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6022378-caf5-43c4-b6e4-783528280fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:59.873639 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6022378-caf5-43c4-b6e4-783528280fca admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:59.884085 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1252/2504] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:19:59.889403 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01600f55-8b69-4f6b-8e36-bc08e0c48cf7 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:59.901771 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01600f55-8b69-4f6b-8e36-bc08e0c48cf7 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:59.909864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2df211-38c8-44c9-a183-e851bde0f924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:59.909864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2df211-38c8-44c9-a183-e851bde0f924 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:59.909864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2df211-38c8-44c9-a183-e851bde0f924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:59.909864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2df211-38c8-44c9-a183-e851bde0f924 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:59.910500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2df211-38c8-44c9-a183-e851bde0f924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:59.910681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2df211-38c8-44c9-a183-e851bde0f924 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:19:59.922839 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1253/2505] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:19:59.927966 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6022378-caf5-43c4-b6e4-783528280fca admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:19:59.942128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6022378-caf5-43c4-b6e4-783528280fca admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:19:59.952520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332efa03-96e8-4012-8a85-d9d339864f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:19:59.954164 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332efa03-96e8-4012-8a85-d9d339864f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:19:59.954471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332efa03-96e8-4012-8a85-d9d339864f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:19:59.954809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332efa03-96e8-4012-8a85-d9d339864f67 admin admin] PATH_INFO: `/v3/projects/352fb8e430e14872b4c6dd99d5e4c01e/users/ad893d47aeb948a9b09b717e609a6577/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:19:59.976492 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332efa03-96e8-4012-8a85-d9d339864f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=352fb8e430e14872b4c6dd99d5e4c01e, user_id=ad893d47aeb948a9b09b717e609a6577, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:19:59.979565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332efa03-96e8-4012-8a85-d9d339864f67 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:00.075250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1253/2506] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:19:59 2022] PUT /identity/v3/projects/352fb8e430e14872b4c6dd99d5e4c01e/users/ad893d47aeb948a9b09b717e609a6577/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:00.096128 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21819435-3447-4669-8e9a-7b83328c98dc None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:00.096482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21819435-3447-4669-8e9a-7b83328c98dc None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:00.096927 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21819435-3447-4669-8e9a-7b83328c98dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:00.150324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-21819435-3447-4669-8e9a-7b83328c98dc None None] MFA Rules not processed for user `ad893d47aeb948a9b09b717e609a6577`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:00.152500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-21819435-3447-4669-8e9a-7b83328c98dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:00.254273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1254/2507] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:20:00.259567 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332efa03-96e8-4012-8a85-d9d339864f67 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:00.335502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332efa03-96e8-4012-8a85-d9d339864f67 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:00.340821 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ef26d-5682-45a7-9d2b-d54cdf9fa8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:00.341529 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ef26d-5682-45a7-9d2b-d54cdf9fa8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:00.341819 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ef26d-5682-45a7-9d2b-d54cdf9fa8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:00.342110 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ef26d-5682-45a7-9d2b-d54cdf9fa8d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:00.342874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ef26d-5682-45a7-9d2b-d54cdf9fa8d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:00.344912 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ef26d-5682-45a7-9d2b-d54cdf9fa8d3 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:00.440240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1254/2508] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:20:00 2022] POST /identity/v3/projects => generated 379 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:20:00.444485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21819435-3447-4669-8e9a-7b83328c98dc None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:00.456831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21819435-3447-4669-8e9a-7b83328c98dc None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:00.529117 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cf17e-d0be-4be4-8a51-3ae9ce14b04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:00.529815 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cf17e-d0be-4be4-8a51-3ae9ce14b04e admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:00.530106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cf17e-d0be-4be4-8a51-3ae9ce14b04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:00.530388 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cf17e-d0be-4be4-8a51-3ae9ce14b04e admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:00.531140 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93cf17e-d0be-4be4-8a51-3ae9ce14b04e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:00.533150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93cf17e-d0be-4be4-8a51-3ae9ce14b04e admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:00.629790 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1255/2509] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:20:00 2022] POST /identity/v3/users => generated 296 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:20:00.636015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ef26d-5682-45a7-9d2b-d54cdf9fa8d3 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:00.647799 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ef26d-5682-45a7-9d2b-d54cdf9fa8d3 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:00.654053 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df9a1c-8dab-473d-b397-39c9da25a38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:00.655084 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df9a1c-8dab-473d-b397-39c9da25a38d admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:00.656291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df9a1c-8dab-473d-b397-39c9da25a38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:00.656484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df9a1c-8dab-473d-b397-39c9da25a38d admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:00.657610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df9a1c-8dab-473d-b397-39c9da25a38d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:00.659871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df9a1c-8dab-473d-b397-39c9da25a38d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:00.672459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1255/2510] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:20:00.677282 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cf17e-d0be-4be4-8a51-3ae9ce14b04e admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:00.691099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cf17e-d0be-4be4-8a51-3ae9ce14b04e admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:00.700482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f2394-032a-47c6-a15b-ade61e5af76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:00.702232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f2394-032a-47c6-a15b-ade61e5af76e admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:00.702879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f2394-032a-47c6-a15b-ade61e5af76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:00.703490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f2394-032a-47c6-a15b-ade61e5af76e admin admin] PATH_INFO: `/v3/projects/42eebf9b93104a95aeaf621d51769bad/users/11a76fec1ee54641ac9a7185ad2c0987/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:00.731115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f2394-032a-47c6-a15b-ade61e5af76e admin admin] RBAC: Authorizing `identity:create_grant(project_id=42eebf9b93104a95aeaf621d51769bad, user_id=11a76fec1ee54641ac9a7185ad2c0987, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:00.733911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f2394-032a-47c6-a15b-ade61e5af76e admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:00.810141 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:00.816842 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1256/2511] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:20:00 2022] PUT /identity/v3/projects/42eebf9b93104a95aeaf621d51769bad/users/11a76fec1ee54641ac9a7185ad2c0987/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:20:00.823097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df9a1c-8dab-473d-b397-39c9da25a38d admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:00.842115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df9a1c-8dab-473d-b397-39c9da25a38d admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:00.881063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:00.928393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10540cd6-a5ef-466a-9378-05bbc43f23c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:00.929323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10540cd6-a5ef-466a-9378-05bbc43f23c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:00.929629 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10540cd6-a5ef-466a-9378-05bbc43f23c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:00.929965 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10540cd6-a5ef-466a-9378-05bbc43f23c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:00.930859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10540cd6-a5ef-466a-9378-05bbc43f23c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:00.932624 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10540cd6-a5ef-466a-9378-05bbc43f23c1 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:00.946194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1256/2512] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:20:00.951803 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f2394-032a-47c6-a15b-ade61e5af76e admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:00.964285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f2394-032a-47c6-a15b-ade61e5af76e admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:00.969575 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d29c8-1013-49fe-a4f1-3766995df0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:00.970794 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d29c8-1013-49fe-a4f1-3766995df0e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:00.971085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d29c8-1013-49fe-a4f1-3766995df0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:00.971375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d29c8-1013-49fe-a4f1-3766995df0e8 admin admin] PATH_INFO: `/v3/projects/42eebf9b93104a95aeaf621d51769bad/users/11a76fec1ee54641ac9a7185ad2c0987/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:00.973650 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d29c8-1013-49fe-a4f1-3766995df0e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42eebf9b93104a95aeaf621d51769bad, user_id=11a76fec1ee54641ac9a7185ad2c0987, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:00.977064 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d29c8-1013-49fe-a4f1-3766995df0e8 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:00.995983 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1257/2513] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:20:00 2022] PUT /identity/v3/projects/42eebf9b93104a95aeaf621d51769bad/users/11a76fec1ee54641ac9a7185ad2c0987/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:01.010849 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e847cd-7a9c-4ec7-9f47-8c8b7de85354 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:01.011570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e847cd-7a9c-4ec7-9f47-8c8b7de85354 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:01.011570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e847cd-7a9c-4ec7-9f47-8c8b7de85354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:01.044540 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-69e847cd-7a9c-4ec7-9f47-8c8b7de85354 None None] MFA Rules not processed for user `11a76fec1ee54641ac9a7185ad2c0987`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:01.046269 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-69e847cd-7a9c-4ec7-9f47-8c8b7de85354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:01.128697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1257/2514] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:01.144963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d29c8-1013-49fe-a4f1-3766995df0e8 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:01.146915 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-426d29c8-1013-49fe-a4f1-3766995df0e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:01.215353 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d29c8-1013-49fe-a4f1-3766995df0e8 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:01.219144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929757a-0b72-4e47-af28-2c8ce7af1d60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:01.219822 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929757a-0b72-4e47-af28-2c8ce7af1d60 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:01.220110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929757a-0b72-4e47-af28-2c8ce7af1d60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:01.220649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929757a-0b72-4e47-af28-2c8ce7af1d60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:01.232138 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929757a-0b72-4e47-af28-2c8ce7af1d60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:01.233611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929757a-0b72-4e47-af28-2c8ce7af1d60 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:01.249175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1258/2515] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:20:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:20:01.909090 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e847cd-7a9c-4ec7-9f47-8c8b7de85354 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:02.015569 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e847cd-7a9c-4ec7-9f47-8c8b7de85354 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:02.024035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ddb3d-b5b8-4021-9bbb-7e3d63b287a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:02.025080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ddb3d-b5b8-4021-9bbb-7e3d63b287a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:02.025453 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ddb3d-b5b8-4021-9bbb-7e3d63b287a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:02.025814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ddb3d-b5b8-4021-9bbb-7e3d63b287a4 admin admin] PATH_INFO: `/v3/users/ad893d47aeb948a9b09b717e609a6577` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:02.027238 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ddb3d-b5b8-4021-9bbb-7e3d63b287a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad893d47aeb948a9b09b717e609a6577)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:02.029423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ddb3d-b5b8-4021-9bbb-7e3d63b287a4 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:02.266273 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2e6ddb3d-b5b8-4021-9bbb-7e3d63b287a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad893d47aeb948a9b09b717e609a6577'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:02.272618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:02.277782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2e6ddb3d-b5b8-4021-9bbb-7e3d63b287a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad893d47aeb948a9b09b717e609a6577'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:02.278905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2e6ddb3d-b5b8-4021-9bbb-7e3d63b287a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad893d47aeb948a9b09b717e609a6577'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:02.296779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2e6ddb3d-b5b8-4021-9bbb-7e3d63b287a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad893d47aeb948a9b09b717e609a6577'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:02.392584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-2e6ddb3d-b5b8-4021-9bbb-7e3d63b287a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad893d47aeb948a9b09b717e609a6577'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:02.410395 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1258/2516] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:20:01 2022] DELETE /identity/v3/users/ad893d47aeb948a9b09b717e609a6577 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:02.413661 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f38faf-a68e-4974-a7ec-bec578afb39a None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:02.414666 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f38faf-a68e-4974-a7ec-bec578afb39a None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:02.414666 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f38faf-a68e-4974-a7ec-bec578afb39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:02.501930 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-15f38faf-a68e-4974-a7ec-bec578afb39a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:02.501930 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-15f38faf-a68e-4974-a7ec-bec578afb39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:02.547725 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1259/2517] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:20:02.551561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ddb3d-b5b8-4021-9bbb-7e3d63b287a4 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:02.553041 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-2e6ddb3d-b5b8-4021-9bbb-7e3d63b287a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:02.623644 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ddb3d-b5b8-4021-9bbb-7e3d63b287a4 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:02.627583 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2517f5-eef8-4f3c-a595-55a2d98dc48a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:02.628242 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2517f5-eef8-4f3c-a595-55a2d98dc48a service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:02.628528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2517f5-eef8-4f3c-a595-55a2d98dc48a service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:02.628871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2517f5-eef8-4f3c-a595-55a2d98dc48a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:02.640140 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2517f5-eef8-4f3c-a595-55a2d98dc48a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:02.641629 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2517f5-eef8-4f3c-a595-55a2d98dc48a service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:02.653728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1259/2518] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:20:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:20:03.731589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f38faf-a68e-4974-a7ec-bec578afb39a None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:03.734061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-15f38faf-a68e-4974-a7ec-bec578afb39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:03.766871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f38faf-a68e-4974-a7ec-bec578afb39a None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:03.774377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:03.776020 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:03.783629 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:03.783629 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6 admin admin] PATH_INFO: `/v3/projects/352fb8e430e14872b4c6dd99d5e4c01e` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:03.783629 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=352fb8e430e14872b4c6dd99d5e4c01e)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:03.783629 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:03.950117 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '352fb8e430e14872b4c6dd99d5e4c01e', 'request_id': 'req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:03.957991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '352fb8e430e14872b4c6dd99d5e4c01e', 'request_id': 'req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:03.965164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '352fb8e430e14872b4c6dd99d5e4c01e', 'request_id': 'req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:04.067080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6 admin admin] The token cache is being invalidate because project 352fb8e430e14872b4c6dd99d5e4c01e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:20:04.067411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:04.068414 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:04.070626 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1260/2519] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:20:03 2022] DELETE /identity/v3/projects/352fb8e430e14872b4c6dd99d5e4c01e => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:04.075209 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2517f5-eef8-4f3c-a595-55a2d98dc48a service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:04.085788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2517f5-eef8-4f3c-a595-55a2d98dc48a service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:04.161124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e753b-c0c2-4cab-9dc5-131812075ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:04.162097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e753b-c0c2-4cab-9dc5-131812075ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:04.162509 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e753b-c0c2-4cab-9dc5-131812075ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:04.162885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e753b-c0c2-4cab-9dc5-131812075ecb admin admin] PATH_INFO: `/v3/users/11a76fec1ee54641ac9a7185ad2c0987` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:04.165916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e753b-c0c2-4cab-9dc5-131812075ecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=11a76fec1ee54641ac9a7185ad2c0987)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:04.168184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e753b-c0c2-4cab-9dc5-131812075ecb admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:04.417279 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e90e753b-c0c2-4cab-9dc5-131812075ecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11a76fec1ee54641ac9a7185ad2c0987'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:04.425129 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e90e753b-c0c2-4cab-9dc5-131812075ecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11a76fec1ee54641ac9a7185ad2c0987'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:04.425890 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e90e753b-c0c2-4cab-9dc5-131812075ecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11a76fec1ee54641ac9a7185ad2c0987'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:04.440335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e90e753b-c0c2-4cab-9dc5-131812075ecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11a76fec1ee54641ac9a7185ad2c0987'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:04.442647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:04.545965 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e90e753b-c0c2-4cab-9dc5-131812075ecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11a76fec1ee54641ac9a7185ad2c0987'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:04.559039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1260/2520] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:20:04 2022] DELETE /identity/v3/users/11a76fec1ee54641ac9a7185ad2c0987 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:05.270043 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:05.271871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:05.286971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bb042-8ac3-4c73-ab00-d6706e4580d6 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:05.296876 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:05.360696 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be4a7b-23e8-40ef-8f92-07f35b5883d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:05.361643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be4a7b-23e8-40ef-8f92-07f35b5883d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:05.361931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be4a7b-23e8-40ef-8f92-07f35b5883d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:05.362220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be4a7b-23e8-40ef-8f92-07f35b5883d1 admin admin] PATH_INFO: `/v3/projects/42eebf9b93104a95aeaf621d51769bad` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:05.363629 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be4a7b-23e8-40ef-8f92-07f35b5883d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42eebf9b93104a95aeaf621d51769bad)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:05.365466 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be4a7b-23e8-40ef-8f92-07f35b5883d1 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:05.494983 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-21be4a7b-23e8-40ef-8f92-07f35b5883d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42eebf9b93104a95aeaf621d51769bad', 'request_id': 'req-21be4a7b-23e8-40ef-8f92-07f35b5883d1'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:05.502162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-21be4a7b-23e8-40ef-8f92-07f35b5883d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42eebf9b93104a95aeaf621d51769bad', 'request_id': 'req-21be4a7b-23e8-40ef-8f92-07f35b5883d1'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:05.508271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-21be4a7b-23e8-40ef-8f92-07f35b5883d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42eebf9b93104a95aeaf621d51769bad', 'request_id': 'req-21be4a7b-23e8-40ef-8f92-07f35b5883d1'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:05.627461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-21be4a7b-23e8-40ef-8f92-07f35b5883d1 admin admin] The token cache is being invalidate because project 42eebf9b93104a95aeaf621d51769bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:20:05.628366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-21be4a7b-23e8-40ef-8f92-07f35b5883d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:05.629161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-21be4a7b-23e8-40ef-8f92-07f35b5883d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:05.636048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1261/2521] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:20:05 2022] DELETE /identity/v3/projects/42eebf9b93104a95aeaf621d51769bad => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:05.638027 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30360b1-38dd-4c0a-bf76-f157e04136fa None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:05.638790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30360b1-38dd-4c0a-bf76-f157e04136fa None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:05.638894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30360b1-38dd-4c0a-bf76-f157e04136fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:05.663289 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-b30360b1-38dd-4c0a-bf76-f157e04136fa None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:05.665273 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b30360b1-38dd-4c0a-bf76-f157e04136fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:05.755170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1261/2522] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:20:05.760498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23698ae-0e8a-42fd-9dcf-57ef26bb9b1f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:05.761516 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23698ae-0e8a-42fd-9dcf-57ef26bb9b1f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:05.761516 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23698ae-0e8a-42fd-9dcf-57ef26bb9b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:05.783529 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-f23698ae-0e8a-42fd-9dcf-57ef26bb9b1f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:05.784778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-f23698ae-0e8a-42fd-9dcf-57ef26bb9b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:05.794452 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1262/2523] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:20:05.798934 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30360b1-38dd-4c0a-bf76-f157e04136fa None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:05.809556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30360b1-38dd-4c0a-bf76-f157e04136fa None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:05.814731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8ed5a-ae5a-430a-b336-f99ac5f700f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:05.815412 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8ed5a-ae5a-430a-b336-f99ac5f700f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:05.815699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8ed5a-ae5a-430a-b336-f99ac5f700f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:05.815983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8ed5a-ae5a-430a-b336-f99ac5f700f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:05.816759 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8ed5a-ae5a-430a-b336-f99ac5f700f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:20:05.817073 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8ed5a-ae5a-430a-b336-f99ac5f700f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:05.818783 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8ed5a-ae5a-430a-b336-f99ac5f700f4 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:05.832186 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1262/2524] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:20:05.837298 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23698ae-0e8a-42fd-9dcf-57ef26bb9b1f None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:05.847465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23698ae-0e8a-42fd-9dcf-57ef26bb9b1f None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:05.853277 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee075d-1e75-4061-a925-04058c571aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:05.854590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee075d-1e75-4061-a925-04058c571aaf admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:05.854590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee075d-1e75-4061-a925-04058c571aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:05.854954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee075d-1e75-4061-a925-04058c571aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:05.855558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee075d-1e75-4061-a925-04058c571aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:05.857516 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee075d-1e75-4061-a925-04058c571aaf admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:06.103641 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1263/2525] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:20:05 2022] POST /identity/v3/projects => generated 383 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:20:06.108074 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8ed5a-ae5a-430a-b336-f99ac5f700f4 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:06.119460 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8ed5a-ae5a-430a-b336-f99ac5f700f4 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:06.185128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd75fb-ea72-42a1-888c-2968025b2b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:06.185810 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd75fb-ea72-42a1-888c-2968025b2b6f admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:06.186097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd75fb-ea72-42a1-888c-2968025b2b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:06.186381 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd75fb-ea72-42a1-888c-2968025b2b6f admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:06.187136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd75fb-ea72-42a1-888c-2968025b2b6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:06.189211 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd75fb-ea72-42a1-888c-2968025b2b6f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:06.299735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1263/2526] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:20:06 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:20:06.304693 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee075d-1e75-4061-a925-04058c571aaf admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:06.316923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee075d-1e75-4061-a925-04058c571aaf admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:06.323586 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94d13b-0283-4900-a2f8-04e51524483b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:06.324294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94d13b-0283-4900-a2f8-04e51524483b admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:06.324736 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94d13b-0283-4900-a2f8-04e51524483b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:06.325026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94d13b-0283-4900-a2f8-04e51524483b admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:06.325858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94d13b-0283-4900-a2f8-04e51524483b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:06.327748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94d13b-0283-4900-a2f8-04e51524483b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:06.340627 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1264/2527] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:20:06.349913 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd75fb-ea72-42a1-888c-2968025b2b6f admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:06.364154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd75fb-ea72-42a1-888c-2968025b2b6f admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:06.370878 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365e87b-f96b-41a2-b4e0-fa20df1e5d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:06.375253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365e87b-f96b-41a2-b4e0-fa20df1e5d26 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:06.375253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365e87b-f96b-41a2-b4e0-fa20df1e5d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:06.375253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365e87b-f96b-41a2-b4e0-fa20df1e5d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:06.375253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365e87b-f96b-41a2-b4e0-fa20df1e5d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:06.375253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365e87b-f96b-41a2-b4e0-fa20df1e5d26 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:06.389073 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1264/2528] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:20:06.395202 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94d13b-0283-4900-a2f8-04e51524483b admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:06.415398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94d13b-0283-4900-a2f8-04e51524483b admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:06.420535 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216dbe54-b9bb-463d-93ed-2a7887a42751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:06.421438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dbe54-b9bb-463d-93ed-2a7887a42751 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:06.421714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dbe54-b9bb-463d-93ed-2a7887a42751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:06.421994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dbe54-b9bb-463d-93ed-2a7887a42751 admin admin] PATH_INFO: `/v3/projects/44df4d07a30a455eb5aceb5d14572abb/users/2f4e857441164452a31691354f9d8fc4/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:06.446088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216dbe54-b9bb-463d-93ed-2a7887a42751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44df4d07a30a455eb5aceb5d14572abb, user_id=2f4e857441164452a31691354f9d8fc4, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:06.448862 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216dbe54-b9bb-463d-93ed-2a7887a42751 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:06.546855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1265/2529] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:20:06 2022] PUT /identity/v3/projects/44df4d07a30a455eb5aceb5d14572abb/users/2f4e857441164452a31691354f9d8fc4/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:20:06.560649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90831a10-3dac-40c5-b0a3-c8a1ef3a97c0 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:06.561159 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90831a10-3dac-40c5-b0a3-c8a1ef3a97c0 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:06.561656 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90831a10-3dac-40c5-b0a3-c8a1ef3a97c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:06.570768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f19b13-2b9a-4f07-8f72-9cfaf09cd81e None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:06.571257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f19b13-2b9a-4f07-8f72-9cfaf09cd81e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:06.571752 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f19b13-2b9a-4f07-8f72-9cfaf09cd81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:06.600718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-a3f19b13-2b9a-4f07-8f72-9cfaf09cd81e None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:06.605331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a3f19b13-2b9a-4f07-8f72-9cfaf09cd81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:06.607734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-90831a10-3dac-40c5-b0a3-c8a1ef3a97c0 None None] MFA Rules not processed for user `2f4e857441164452a31691354f9d8fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:06.609030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-90831a10-3dac-40c5-b0a3-c8a1ef3a97c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:06.661189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:06.667761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:06.703299 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1265/2530] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:20:06.708153 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90831a10-3dac-40c5-b0a3-c8a1ef3a97c0 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:06.708899 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1266/2531] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Jul 18 15:20:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:20:06.795075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90831a10-3dac-40c5-b0a3-c8a1ef3a97c0 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:06.799995 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9459807-8ee1-4b3e-83df-00afb0dd2be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:06.800677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9459807-8ee1-4b3e-83df-00afb0dd2be9 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:06.800966 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9459807-8ee1-4b3e-83df-00afb0dd2be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:06.801250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9459807-8ee1-4b3e-83df-00afb0dd2be9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:06.801952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9459807-8ee1-4b3e-83df-00afb0dd2be9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:06.804495 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9459807-8ee1-4b3e-83df-00afb0dd2be9 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:06.893721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f19b13-2b9a-4f07-8f72-9cfaf09cd81e None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:06.895253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a3f19b13-2b9a-4f07-8f72-9cfaf09cd81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:06.950948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1266/2532] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:20:06 2022] POST /identity/v3/projects => generated 383 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:20:06.956105 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9459807-8ee1-4b3e-83df-00afb0dd2be9 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:07.001755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f19b13-2b9a-4f07-8f72-9cfaf09cd81e None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:07.058858 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9459807-8ee1-4b3e-83df-00afb0dd2be9 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:07.068360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a8b3c-089d-4022-9f75-c78a57c0855d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:07.068640 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a8b3c-089d-4022-9f75-c78a57c0855d admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:07.069056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a8b3c-089d-4022-9f75-c78a57c0855d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:07.069547 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a8b3c-089d-4022-9f75-c78a57c0855d admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:07.070833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a8b3c-089d-4022-9f75-c78a57c0855d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:07.073510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a8b3c-089d-4022-9f75-c78a57c0855d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:07.087715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6eea9-9f61-4ca3-a4a3-04e71b4ad6eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:07.088503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6eea9-9f61-4ca3-a4a3-04e71b4ad6eb service swift] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:07.088788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6eea9-9f61-4ca3-a4a3-04e71b4ad6eb service swift] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:07.089108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6eea9-9f61-4ca3-a4a3-04e71b4ad6eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:07.196924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1267/2533] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:20:06 2022] POST /identity/v3/users => generated 298 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:20:07.201004 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a8b3c-089d-4022-9f75-c78a57c0855d admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:07.204414 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6eea9-9f61-4ca3-a4a3-04e71b4ad6eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:07.206255 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6eea9-9f61-4ca3-a4a3-04e71b4ad6eb service swift] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:07.219372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a8b3c-089d-4022-9f75-c78a57c0855d admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:07.224091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1267/2534] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:20:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 15:20:07.225414 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74644056-80cc-4427-ba20-f0f266ff376b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:07.226798 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74644056-80cc-4427-ba20-f0f266ff376b admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:07.227460 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74644056-80cc-4427-ba20-f0f266ff376b admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:07.227804 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74644056-80cc-4427-ba20-f0f266ff376b admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:07.229236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74644056-80cc-4427-ba20-f0f266ff376b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:07.231529 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74644056-80cc-4427-ba20-f0f266ff376b admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:07.250965 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1268/2535] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:20:07.255334 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6eea9-9f61-4ca3-a4a3-04e71b4ad6eb service swift] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:07.269460 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6eea9-9f61-4ca3-a4a3-04e71b4ad6eb service swift] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:07.291738 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb90942-b95f-4eb9-9819-5784b7eb7b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:07.292730 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb90942-b95f-4eb9-9819-5784b7eb7b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:07.293117 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb90942-b95f-4eb9-9819-5784b7eb7b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:07.293400 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb90942-b95f-4eb9-9819-5784b7eb7b33 admin admin] PATH_INFO: `/v3/projects/11488561a3d54edd948f3ddfc6245fa0/users/b3ea1daba2234f6aaf40566cd41549cd/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:07.317992 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb90942-b95f-4eb9-9819-5784b7eb7b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11488561a3d54edd948f3ddfc6245fa0, user_id=b3ea1daba2234f6aaf40566cd41549cd, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:07.321435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb90942-b95f-4eb9-9819-5784b7eb7b33 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:07.638105 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1268/2536] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:20:07 2022] PUT /identity/v3/projects/11488561a3d54edd948f3ddfc6245fa0/users/b3ea1daba2234f6aaf40566cd41549cd/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:07.639552 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74644056-80cc-4427-ba20-f0f266ff376b admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:07.659287 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74644056-80cc-4427-ba20-f0f266ff376b admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:07.761302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:07.780595 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9788e-e271-4826-8950-755fc9c2fa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:07.781401 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9788e-e271-4826-8950-755fc9c2fa2d admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:07.781830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9788e-e271-4826-8950-755fc9c2fa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:07.782196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9788e-e271-4826-8950-755fc9c2fa2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:07.783391 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9788e-e271-4826-8950-755fc9c2fa2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:07.785363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9788e-e271-4826-8950-755fc9c2fa2d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:07.820757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1269/2537] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:20:07.822212 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb90942-b95f-4eb9-9819-5784b7eb7b33 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:07.829829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:07.838154 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb90942-b95f-4eb9-9819-5784b7eb7b33 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:07.873522 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5ee95-5017-4d01-a144-9049c5236ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:07.873522 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5ee95-5017-4d01-a144-9049c5236ef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:07.873522 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5ee95-5017-4d01-a144-9049c5236ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:07.873522 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5ee95-5017-4d01-a144-9049c5236ef7 admin admin] PATH_INFO: `/v3/projects/11488561a3d54edd948f3ddfc6245fa0/users/b3ea1daba2234f6aaf40566cd41549cd/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:07.874664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5ee95-5017-4d01-a144-9049c5236ef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11488561a3d54edd948f3ddfc6245fa0, user_id=b3ea1daba2234f6aaf40566cd41549cd, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:07.874664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5ee95-5017-4d01-a144-9049c5236ef7 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:07.874664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1269/2538] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:20:07 2022] PUT /identity/v3/projects/11488561a3d54edd948f3ddfc6245fa0/users/b3ea1daba2234f6aaf40566cd41549cd/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:07.890937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2d49d-d1ca-4185-b67e-daf788830f7f None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:07.891775 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2d49d-d1ca-4185-b67e-daf788830f7f None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:07.892163 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2d49d-d1ca-4185-b67e-daf788830f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:07.949982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-f7e2d49d-d1ca-4185-b67e-daf788830f7f None None] MFA Rules not processed for user `b3ea1daba2234f6aaf40566cd41549cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:07.951435 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f7e2d49d-d1ca-4185-b67e-daf788830f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:08.061892 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1270/2539] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:08.077563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5ee95-5017-4d01-a144-9049c5236ef7 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:08.079570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-8fc5ee95-5017-4d01-a144-9049c5236ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:08.188459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5ee95-5017-4d01-a144-9049c5236ef7 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:08.192812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b858e-1096-4e9d-983b-ce2fbeaddb83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:08.193775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b858e-1096-4e9d-983b-ce2fbeaddb83 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:08.194304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b858e-1096-4e9d-983b-ce2fbeaddb83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:08.195290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b858e-1096-4e9d-983b-ce2fbeaddb83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:08.206722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b858e-1096-4e9d-983b-ce2fbeaddb83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:08.211805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b858e-1096-4e9d-983b-ce2fbeaddb83 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:08.225294 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1270/2540] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:20:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:20:08.421128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2d49d-d1ca-4185-b67e-daf788830f7f None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:08.433163 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2d49d-d1ca-4185-b67e-daf788830f7f None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:08.438077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863eb15-69bf-443d-abca-c400a6048637 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:08.438806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863eb15-69bf-443d-abca-c400a6048637 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:08.439100 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863eb15-69bf-443d-abca-c400a6048637 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:08.439386 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863eb15-69bf-443d-abca-c400a6048637 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:08.577978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863eb15-69bf-443d-abca-c400a6048637 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:08.581891 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863eb15-69bf-443d-abca-c400a6048637 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:08.599735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1271/2541] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:20:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:20:09.561088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b858e-1096-4e9d-983b-ce2fbeaddb83 service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:09.650455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b858e-1096-4e9d-983b-ce2fbeaddb83 service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:09.677477 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa6b59-67ba-4376-8970-1fb6eadda252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:09.678512 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa6b59-67ba-4376-8970-1fb6eadda252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:09.685478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa6b59-67ba-4376-8970-1fb6eadda252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:09.685478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa6b59-67ba-4376-8970-1fb6eadda252 admin admin] PATH_INFO: `/v3/users/2f4e857441164452a31691354f9d8fc4` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:09.685478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa6b59-67ba-4376-8970-1fb6eadda252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f4e857441164452a31691354f9d8fc4)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:09.688784 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa6b59-67ba-4376-8970-1fb6eadda252 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:09.954150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-1efa6b59-67ba-4376-8970-1fb6eadda252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f4e857441164452a31691354f9d8fc4'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:09.955289 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-1efa6b59-67ba-4376-8970-1fb6eadda252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f4e857441164452a31691354f9d8fc4'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:09.956338 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-1efa6b59-67ba-4376-8970-1fb6eadda252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f4e857441164452a31691354f9d8fc4'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:10.033034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-1efa6b59-67ba-4376-8970-1fb6eadda252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f4e857441164452a31691354f9d8fc4'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:10.062091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-1efa6b59-67ba-4376-8970-1fb6eadda252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f4e857441164452a31691354f9d8fc4'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:10.082888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1271/2542] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:20:09 2022] DELETE /identity/v3/users/2f4e857441164452a31691354f9d8fc4 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:10.084306 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53c611-4d6a-40d9-bd97-58b9a540e9d1 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:10.085112 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53c611-4d6a-40d9-bd97-58b9a540e9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:10.085112 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53c611-4d6a-40d9-bd97-58b9a540e9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:10.119810 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-4e53c611-4d6a-40d9-bd97-58b9a540e9d1 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:10.121143 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4e53c611-4d6a-40d9-bd97-58b9a540e9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:10.241029 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1272/2543] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:10.251507 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa6b59-67ba-4376-8970-1fb6eadda252 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:10.253147 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-1efa6b59-67ba-4376-8970-1fb6eadda252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:10.275706 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:10.371173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa6b59-67ba-4376-8970-1fb6eadda252 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:10.377022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd12599-eb88-4081-bc5e-d9bc2691df41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:10.377919 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd12599-eb88-4081-bc5e-d9bc2691df41 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:10.378206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd12599-eb88-4081-bc5e-d9bc2691df41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:10.378510 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd12599-eb88-4081-bc5e-d9bc2691df41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:10.389276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd12599-eb88-4081-bc5e-d9bc2691df41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:10.392887 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd12599-eb88-4081-bc5e-d9bc2691df41 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:10.404434 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1272/2544] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:20:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:20:11.087066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53c611-4d6a-40d9-bd97-58b9a540e9d1 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:11.089458 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4e53c611-4d6a-40d9-bd97-58b9a540e9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:11.102661 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53c611-4d6a-40d9-bd97-58b9a540e9d1 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:11.110605 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451b863-c49e-48f5-b4cf-78c1158ccdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:11.128424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451b863-c49e-48f5-b4cf-78c1158ccdde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:11.128424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451b863-c49e-48f5-b4cf-78c1158ccdde admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:11.128424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451b863-c49e-48f5-b4cf-78c1158ccdde admin admin] PATH_INFO: `/v3/projects/44df4d07a30a455eb5aceb5d14572abb` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:11.128424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451b863-c49e-48f5-b4cf-78c1158ccdde admin admin] RBAC: Authorizing `identity:delete_project(project_id=44df4d07a30a455eb5aceb5d14572abb)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:11.128424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451b863-c49e-48f5-b4cf-78c1158ccdde admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:11.128424 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:11.246000 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f451b863-c49e-48f5-b4cf-78c1158ccdde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44df4d07a30a455eb5aceb5d14572abb', 'request_id': 'req-f451b863-c49e-48f5-b4cf-78c1158ccdde'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:11.254385 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f451b863-c49e-48f5-b4cf-78c1158ccdde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44df4d07a30a455eb5aceb5d14572abb', 'request_id': 'req-f451b863-c49e-48f5-b4cf-78c1158ccdde'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:11.330235 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f451b863-c49e-48f5-b4cf-78c1158ccdde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44df4d07a30a455eb5aceb5d14572abb', 'request_id': 'req-f451b863-c49e-48f5-b4cf-78c1158ccdde'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:11.354476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f451b863-c49e-48f5-b4cf-78c1158ccdde admin admin] The token cache is being invalidate because project 44df4d07a30a455eb5aceb5d14572abb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:20:11.354476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f451b863-c49e-48f5-b4cf-78c1158ccdde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:11.354476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f451b863-c49e-48f5-b4cf-78c1158ccdde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:11.354476 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1273/2545] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:20:11 2022] DELETE /identity/v3/projects/44df4d07a30a455eb5aceb5d14572abb => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:11.357469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd12599-eb88-4081-bc5e-d9bc2691df41 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:11.374280 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd12599-eb88-4081-bc5e-d9bc2691df41 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:11.540958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb037d3-85e1-4635-8f8a-588828e72fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:11.544089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb037d3-85e1-4635-8f8a-588828e72fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:11.544089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb037d3-85e1-4635-8f8a-588828e72fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:11.544089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb037d3-85e1-4635-8f8a-588828e72fa2 admin admin] PATH_INFO: `/v3/users/b3ea1daba2234f6aaf40566cd41549cd` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:11.544089 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb037d3-85e1-4635-8f8a-588828e72fa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3ea1daba2234f6aaf40566cd41549cd)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:11.547427 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb037d3-85e1-4635-8f8a-588828e72fa2 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:11.807625 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ecb037d3-85e1-4635-8f8a-588828e72fa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3ea1daba2234f6aaf40566cd41549cd'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:11.808447 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ecb037d3-85e1-4635-8f8a-588828e72fa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3ea1daba2234f6aaf40566cd41549cd'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:11.809124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ecb037d3-85e1-4635-8f8a-588828e72fa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3ea1daba2234f6aaf40566cd41549cd'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:11.940544 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ecb037d3-85e1-4635-8f8a-588828e72fa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3ea1daba2234f6aaf40566cd41549cd'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:11.966210 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ecb037d3-85e1-4635-8f8a-588828e72fa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3ea1daba2234f6aaf40566cd41549cd'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:12.021830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1273/2546] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:20:11 2022] DELETE /identity/v3/users/b3ea1daba2234f6aaf40566cd41549cd => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:12.665527 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451b863-c49e-48f5-b4cf-78c1158ccdde admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:12.668774 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f451b863-c49e-48f5-b4cf-78c1158ccdde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:12.693323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451b863-c49e-48f5-b4cf-78c1158ccdde admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:12.771838 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d47e64f-3e71-4d51-ab63-a69e9327666a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:12.772782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47e64f-3e71-4d51-ab63-a69e9327666a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:12.773890 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47e64f-3e71-4d51-ab63-a69e9327666a admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:12.774179 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47e64f-3e71-4d51-ab63-a69e9327666a admin admin] PATH_INFO: `/v3/projects/11488561a3d54edd948f3ddfc6245fa0` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:12.775549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d47e64f-3e71-4d51-ab63-a69e9327666a admin admin] RBAC: Authorizing `identity:delete_project(project_id=11488561a3d54edd948f3ddfc6245fa0)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:12.781865 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d47e64f-3e71-4d51-ab63-a69e9327666a admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:12.963804 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9d47e64f-3e71-4d51-ab63-a69e9327666a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11488561a3d54edd948f3ddfc6245fa0', 'request_id': 'req-9d47e64f-3e71-4d51-ab63-a69e9327666a'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:12.972223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9d47e64f-3e71-4d51-ab63-a69e9327666a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11488561a3d54edd948f3ddfc6245fa0', 'request_id': 'req-9d47e64f-3e71-4d51-ab63-a69e9327666a'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:13.094833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9d47e64f-3e71-4d51-ab63-a69e9327666a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11488561a3d54edd948f3ddfc6245fa0', 'request_id': 'req-9d47e64f-3e71-4d51-ab63-a69e9327666a'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:13.111945 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9d47e64f-3e71-4d51-ab63-a69e9327666a admin admin] The token cache is being invalidate because project 11488561a3d54edd948f3ddfc6245fa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:20:13.112420 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9d47e64f-3e71-4d51-ab63-a69e9327666a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:13.113272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-9d47e64f-3e71-4d51-ab63-a69e9327666a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:13.119236 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1274/2547] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:20:12 2022] DELETE /identity/v3/projects/11488561a3d54edd948f3ddfc6245fa0 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:13.123171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038b15a-40fc-4c78-b3f9-2ea97a2f3f05 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:13.123702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038b15a-40fc-4c78-b3f9-2ea97a2f3f05 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:13.123799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038b15a-40fc-4c78-b3f9-2ea97a2f3f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:13.150297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d038b15a-40fc-4c78-b3f9-2ea97a2f3f05 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:13.151443 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d038b15a-40fc-4c78-b3f9-2ea97a2f3f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:13.191371 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:13.312589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1274/2548] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:13.322904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1566d-a209-40c5-b6a1-94d2fd0ea6f8 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:13.323338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1566d-a209-40c5-b6a1-94d2fd0ea6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:13.323684 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1566d-a209-40c5-b6a1-94d2fd0ea6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:13.401506 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-33c1566d-a209-40c5-b6a1-94d2fd0ea6f8 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:13.403076 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-33c1566d-a209-40c5-b6a1-94d2fd0ea6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:13.418687 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1275/2549] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:13.419913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038b15a-40fc-4c78-b3f9-2ea97a2f3f05 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:13.433513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038b15a-40fc-4c78-b3f9-2ea97a2f3f05 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:13.446841 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f59c3-f8c7-41f2-9538-50c117a77384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:13.447636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f59c3-f8c7-41f2-9538-50c117a77384 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:13.447998 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f59c3-f8c7-41f2-9538-50c117a77384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:13.448433 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f59c3-f8c7-41f2-9538-50c117a77384 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:13.449248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f59c3-f8c7-41f2-9538-50c117a77384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:20:13.449858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f59c3-f8c7-41f2-9538-50c117a77384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:13.451900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f59c3-f8c7-41f2-9538-50c117a77384 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:13.485428 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1275/2550] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:20:13.491549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1566d-a209-40c5-b6a1-94d2fd0ea6f8 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:13.502278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1566d-a209-40c5-b6a1-94d2fd0ea6f8 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:13.514037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d3445-6f9c-4169-a6f4-bfdb6411da6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:13.514818 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d3445-6f9c-4169-a6f4-bfdb6411da6d admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:13.515677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d3445-6f9c-4169-a6f4-bfdb6411da6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:13.515677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d3445-6f9c-4169-a6f4-bfdb6411da6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:13.516213 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d3445-6f9c-4169-a6f4-bfdb6411da6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:13.518198 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d3445-6f9c-4169-a6f4-bfdb6411da6d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:13.536934 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:13.627453 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1276/2551] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:20:13 2022] POST /identity/v3/projects => generated 359 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:20:13.632472 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f59c3-f8c7-41f2-9538-50c117a77384 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:13.659342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f59c3-f8c7-41f2-9538-50c117a77384 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:13.756527 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cf143-6a05-4b6b-b72c-7d584f8ec1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:13.759454 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cf143-6a05-4b6b-b72c-7d584f8ec1b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:13.760368 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cf143-6a05-4b6b-b72c-7d584f8ec1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:13.761217 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cf143-6a05-4b6b-b72c-7d584f8ec1b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:13.761424 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6cf143-6a05-4b6b-b72c-7d584f8ec1b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:13.763848 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6cf143-6a05-4b6b-b72c-7d584f8ec1b9 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:13.895414 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1276/2552] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:20:13 2022] POST /identity/v3/users => generated 280 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:20:13.897115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d3445-6f9c-4169-a6f4-bfdb6411da6d admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:13.910033 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d3445-6f9c-4169-a6f4-bfdb6411da6d admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:13.918731 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f7e2e-c9ff-4217-9a72-206a80826d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:13.919436 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f7e2e-c9ff-4217-9a72-206a80826d18 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:13.919724 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f7e2e-c9ff-4217-9a72-206a80826d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:13.920007 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f7e2e-c9ff-4217-9a72-206a80826d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:13.920761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f7e2e-c9ff-4217-9a72-206a80826d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:13.922717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f7e2e-c9ff-4217-9a72-206a80826d18 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:13.938757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1277/2553] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:20:13.942493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cf143-6a05-4b6b-b72c-7d584f8ec1b9 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:13.958331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cf143-6a05-4b6b-b72c-7d584f8ec1b9 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:13.965386 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7b004-6d6c-41f0-b829-b2f7fa21a9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:13.972367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7b004-6d6c-41f0-b829-b2f7fa21a9d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:13.972367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7b004-6d6c-41f0-b829-b2f7fa21a9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:13.972367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7b004-6d6c-41f0-b829-b2f7fa21a9d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:13.972367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7b004-6d6c-41f0-b829-b2f7fa21a9d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:13.972367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7b004-6d6c-41f0-b829-b2f7fa21a9d1 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:13.983598 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1277/2554] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:20:13.987462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f7e2e-c9ff-4217-9a72-206a80826d18 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:14.000404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f7e2e-c9ff-4217-9a72-206a80826d18 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:14.006406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfdcdb-b059-496e-ae17-490bb4dcf668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:14.007818 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfdcdb-b059-496e-ae17-490bb4dcf668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:14.008696 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfdcdb-b059-496e-ae17-490bb4dcf668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:14.008696 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfdcdb-b059-496e-ae17-490bb4dcf668 admin admin] PATH_INFO: `/v3/projects/f619cc15c0f24d05a5b36eab20f962b1/users/33d9b959832f4dc29bc3c8e774ec9536/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:14.028638 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dfdcdb-b059-496e-ae17-490bb4dcf668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f619cc15c0f24d05a5b36eab20f962b1, user_id=33d9b959832f4dc29bc3c8e774ec9536, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:14.031831 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dfdcdb-b059-496e-ae17-490bb4dcf668 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:14.151859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1278/2555] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:20:13 2022] PUT /identity/v3/projects/f619cc15c0f24d05a5b36eab20f962b1/users/33d9b959832f4dc29bc3c8e774ec9536/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:20:14.163488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c54f6-01e0-47ca-abf6-f549ca0a4416 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:14.164169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c54f6-01e0-47ca-abf6-f549ca0a4416 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:14.164169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c54f6-01e0-47ca-abf6-f549ca0a4416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:14.195609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-c18c54f6-01e0-47ca-abf6-f549ca0a4416 None None] MFA Rules not processed for user `33d9b959832f4dc29bc3c8e774ec9536`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:14.196934 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c18c54f6-01e0-47ca-abf6-f549ca0a4416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:14.282681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1278/2556] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:14 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:14.325287 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfdcdb-b059-496e-ae17-490bb4dcf668 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:14.327183 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-27dfdcdb-b059-496e-ae17-490bb4dcf668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:14.412479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfdcdb-b059-496e-ae17-490bb4dcf668 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:14.417285 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb2b48-13ae-4a94-aeda-cb5fe65154ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:14.417950 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb2b48-13ae-4a94-aeda-cb5fe65154ba service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:14.418240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb2b48-13ae-4a94-aeda-cb5fe65154ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:14.418521 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb2b48-13ae-4a94-aeda-cb5fe65154ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:14.428379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb2b48-13ae-4a94-aeda-cb5fe65154ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:14.432398 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb2b48-13ae-4a94-aeda-cb5fe65154ba service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:14.445882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1279/2557] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:20:14 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:20:14.463364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c54f6-01e0-47ca-abf6-f549ca0a4416 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:14.499691 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:14.532131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c54f6-01e0-47ca-abf6-f549ca0a4416 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:14.537007 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781cb046-24fd-49cf-9e85-b6612ca0e18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:14.537827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cb046-24fd-49cf-9e85-b6612ca0e18e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:14.538120 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cb046-24fd-49cf-9e85-b6612ca0e18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:14.539085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cb046-24fd-49cf-9e85-b6612ca0e18e admin admin] PATH_INFO: `/v3/users/33d9b959832f4dc29bc3c8e774ec9536` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:14.540355 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781cb046-24fd-49cf-9e85-b6612ca0e18e admin admin] RBAC: Authorizing `identity:delete_user(user_id=33d9b959832f4dc29bc3c8e774ec9536)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:14.542297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781cb046-24fd-49cf-9e85-b6612ca0e18e admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:14.979560 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-781cb046-24fd-49cf-9e85-b6612ca0e18e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33d9b959832f4dc29bc3c8e774ec9536'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:14.980453 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-781cb046-24fd-49cf-9e85-b6612ca0e18e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33d9b959832f4dc29bc3c8e774ec9536'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:14.987952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-781cb046-24fd-49cf-9e85-b6612ca0e18e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33d9b959832f4dc29bc3c8e774ec9536'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:15.061009 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-781cb046-24fd-49cf-9e85-b6612ca0e18e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33d9b959832f4dc29bc3c8e774ec9536'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:15.083720 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-781cb046-24fd-49cf-9e85-b6612ca0e18e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33d9b959832f4dc29bc3c8e774ec9536'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:15.106519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1279/2558] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:20:14 2022] DELETE /identity/v3/users/33d9b959832f4dc29bc3c8e774ec9536 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:15.111358 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9af54-5d2d-4d86-b49b-4a7a1e3738ca None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:15.112018 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9af54-5d2d-4d86-b49b-4a7a1e3738ca None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:15.112018 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9af54-5d2d-4d86-b49b-4a7a1e3738ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:15.137390 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-b5e9af54-5d2d-4d86-b49b-4a7a1e3738ca None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:15.138732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b5e9af54-5d2d-4d86-b49b-4a7a1e3738ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:15.263320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1280/2559] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:15.273832 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781cb046-24fd-49cf-9e85-b6612ca0e18e admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:15.279580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-781cb046-24fd-49cf-9e85-b6612ca0e18e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:15.377185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781cb046-24fd-49cf-9e85-b6612ca0e18e admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:15.388191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9556a7-ae35-4cae-8d2c-9e59d0f7a3c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:15.389204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9556a7-ae35-4cae-8d2c-9e59d0f7a3c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:15.389683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9556a7-ae35-4cae-8d2c-9e59d0f7a3c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:15.389971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9556a7-ae35-4cae-8d2c-9e59d0f7a3c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:15.400864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9556a7-ae35-4cae-8d2c-9e59d0f7a3c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:15.404090 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9556a7-ae35-4cae-8d2c-9e59d0f7a3c6 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:15.415729 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1280/2560] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:20:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:20:16.106182 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9af54-5d2d-4d86-b49b-4a7a1e3738ca None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:16.109289 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b5e9af54-5d2d-4d86-b49b-4a7a1e3738ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:16.169158 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9af54-5d2d-4d86-b49b-4a7a1e3738ca None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:16.194205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5813a3d-802a-48e1-b5a6-c5e7093a496d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:16.196445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5813a3d-802a-48e1-b5a6-c5e7093a496d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:16.196733 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5813a3d-802a-48e1-b5a6-c5e7093a496d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:16.197289 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5813a3d-802a-48e1-b5a6-c5e7093a496d admin admin] PATH_INFO: `/v3/projects/f619cc15c0f24d05a5b36eab20f962b1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:16.198955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5813a3d-802a-48e1-b5a6-c5e7093a496d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f619cc15c0f24d05a5b36eab20f962b1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:16.201164 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5813a3d-802a-48e1-b5a6-c5e7093a496d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:16.388034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a5813a3d-802a-48e1-b5a6-c5e7093a496d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f619cc15c0f24d05a5b36eab20f962b1', 'request_id': 'req-a5813a3d-802a-48e1-b5a6-c5e7093a496d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:16.403403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a5813a3d-802a-48e1-b5a6-c5e7093a496d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f619cc15c0f24d05a5b36eab20f962b1', 'request_id': 'req-a5813a3d-802a-48e1-b5a6-c5e7093a496d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:16.416765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:16.488153 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a5813a3d-802a-48e1-b5a6-c5e7093a496d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f619cc15c0f24d05a5b36eab20f962b1', 'request_id': 'req-a5813a3d-802a-48e1-b5a6-c5e7093a496d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:16.525097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a5813a3d-802a-48e1-b5a6-c5e7093a496d admin admin] The token cache is being invalidate because project f619cc15c0f24d05a5b36eab20f962b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:20:16.525474 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a5813a3d-802a-48e1-b5a6-c5e7093a496d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:16.526210 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a5813a3d-802a-48e1-b5a6-c5e7093a496d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:16.528283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1281/2561] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:20:16 2022] DELETE /identity/v3/projects/f619cc15c0f24d05a5b36eab20f962b1 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:16.534237 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6c756-20a7-4e2f-999e-f545de7e4708 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:16.536162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6c756-20a7-4e2f-999e-f545de7e4708 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:16.536162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6c756-20a7-4e2f-999e-f545de7e4708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:16.558629 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-76a6c756-20a7-4e2f-999e-f545de7e4708 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:16.560918 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-76a6c756-20a7-4e2f-999e-f545de7e4708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:16.708296 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:16.734043 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1281/2562] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:16.810905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fabd9-66f7-4cab-949a-4b2395387ead None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:16.811173 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fabd9-66f7-4cab-949a-4b2395387ead None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:16.811469 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fabd9-66f7-4cab-949a-4b2395387ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:16.848545 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-8f2fabd9-66f7-4cab-949a-4b2395387ead None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:16.850805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-8f2fabd9-66f7-4cab-949a-4b2395387ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:16.865794 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1282/2563] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:16.871340 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6c756-20a7-4e2f-999e-f545de7e4708 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:16.884691 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6c756-20a7-4e2f-999e-f545de7e4708 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:16.892753 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b1515-a896-4bf6-b58d-2bbd75f47565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:16.893531 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b1515-a896-4bf6-b58d-2bbd75f47565 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:16.898417 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b1515-a896-4bf6-b58d-2bbd75f47565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:16.898417 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b1515-a896-4bf6-b58d-2bbd75f47565 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:16.898417 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b1515-a896-4bf6-b58d-2bbd75f47565 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:20:16.898417 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b1515-a896-4bf6-b58d-2bbd75f47565 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:16.898417 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b1515-a896-4bf6-b58d-2bbd75f47565 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:16.917689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1282/2564] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 15:20:16.918823 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fabd9-66f7-4cab-949a-4b2395387ead None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:16.949471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fabd9-66f7-4cab-949a-4b2395387ead None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:16.954747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd78ca-afb5-48be-81b0-8a90a3ba4379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:16.955455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd78ca-afb5-48be-81b0-8a90a3ba4379 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:16.955901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd78ca-afb5-48be-81b0-8a90a3ba4379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:16.956189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd78ca-afb5-48be-81b0-8a90a3ba4379 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:16.957067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd78ca-afb5-48be-81b0-8a90a3ba4379 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:16.959155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd78ca-afb5-48be-81b0-8a90a3ba4379 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:17.191467 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b1515-a896-4bf6-b58d-2bbd75f47565 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:17.192952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1283/2565] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:20:16 2022] POST /identity/v3/projects => generated 373 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:20:17.209288 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b1515-a896-4bf6-b58d-2bbd75f47565 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:17.331517 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5380b30-cc9d-4b94-956d-a00ecce7e240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:17.332580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5380b30-cc9d-4b94-956d-a00ecce7e240 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:17.332932 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5380b30-cc9d-4b94-956d-a00ecce7e240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:17.333366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5380b30-cc9d-4b94-956d-a00ecce7e240 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:17.334630 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5380b30-cc9d-4b94-956d-a00ecce7e240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:17.342480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5380b30-cc9d-4b94-956d-a00ecce7e240 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:17.483534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1283/2566] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:20:17 2022] POST /identity/v3/users => generated 287 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:20:17.490067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd78ca-afb5-48be-81b0-8a90a3ba4379 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:17.502713 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd78ca-afb5-48be-81b0-8a90a3ba4379 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:17.513195 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a92d64-0da2-41d5-93da-e37e460bfcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:17.514307 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a92d64-0da2-41d5-93da-e37e460bfcae admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:17.514617 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a92d64-0da2-41d5-93da-e37e460bfcae admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:17.515101 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a92d64-0da2-41d5-93da-e37e460bfcae admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:17.515930 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a92d64-0da2-41d5-93da-e37e460bfcae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:17.517677 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a92d64-0da2-41d5-93da-e37e460bfcae admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:17.534836 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1284/2567] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:20:17.539758 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5380b30-cc9d-4b94-956d-a00ecce7e240 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:17.566279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5380b30-cc9d-4b94-956d-a00ecce7e240 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:17.574713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241be27-734b-4688-bfd4-5ce07e73a404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:17.575624 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241be27-734b-4688-bfd4-5ce07e73a404 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:17.576030 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241be27-734b-4688-bfd4-5ce07e73a404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:17.576424 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241be27-734b-4688-bfd4-5ce07e73a404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:17.577297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241be27-734b-4688-bfd4-5ce07e73a404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:17.579260 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241be27-734b-4688-bfd4-5ce07e73a404 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:17.594778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1284/2568] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:20:17.599302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a92d64-0da2-41d5-93da-e37e460bfcae admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:17.615361 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a92d64-0da2-41d5-93da-e37e460bfcae admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:17.622568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac2dc8-1545-438a-99f2-ad05fa779c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:17.624078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac2dc8-1545-438a-99f2-ad05fa779c46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:17.624813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac2dc8-1545-438a-99f2-ad05fa779c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:17.625281 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac2dc8-1545-438a-99f2-ad05fa779c46 admin admin] PATH_INFO: `/v3/projects/20b8703a2b544d6e9504f8fbd7574bff/users/f9cb2fa45c0e4c5abccb89ea88684713/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:17.651552 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac2dc8-1545-438a-99f2-ad05fa779c46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20b8703a2b544d6e9504f8fbd7574bff, user_id=f9cb2fa45c0e4c5abccb89ea88684713, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:17.656283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac2dc8-1545-438a-99f2-ad05fa779c46 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:17.739216 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1285/2569] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:20:17 2022] PUT /identity/v3/projects/20b8703a2b544d6e9504f8fbd7574bff/users/f9cb2fa45c0e4c5abccb89ea88684713/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:17.752058 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4e9fc-8fd2-41ea-80bf-8eef7b2c879b None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:17.753153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4e9fc-8fd2-41ea-80bf-8eef7b2c879b None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:17.753702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4e9fc-8fd2-41ea-80bf-8eef7b2c879b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:17.819048 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-05a4e9fc-8fd2-41ea-80bf-8eef7b2c879b None None] MFA Rules not processed for user `f9cb2fa45c0e4c5abccb89ea88684713`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:17.820793 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-05a4e9fc-8fd2-41ea-80bf-8eef7b2c879b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:17.943236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1285/2570] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:17.950743 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac2dc8-1545-438a-99f2-ad05fa779c46 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:18.063679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac2dc8-1545-438a-99f2-ad05fa779c46 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:18.072974 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7daae11-9f91-477b-b422-82f7a2b7cc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:18.074130 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daae11-9f91-477b-b422-82f7a2b7cc4d admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:18.074684 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daae11-9f91-477b-b422-82f7a2b7cc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:18.075227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daae11-9f91-477b-b422-82f7a2b7cc4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:18.076519 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7daae11-9f91-477b-b422-82f7a2b7cc4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:18.080366 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7daae11-9f91-477b-b422-82f7a2b7cc4d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:18.219775 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1286/2571] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:20:17 2022] POST /identity/v3/projects => generated 375 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:20:18.223926 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4e9fc-8fd2-41ea-80bf-8eef7b2c879b None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:18.238782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4e9fc-8fd2-41ea-80bf-8eef7b2c879b None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:18.310435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:18.325835 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499cae1-6162-4eb4-986a-3d2451693b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:18.326563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499cae1-6162-4eb4-986a-3d2451693b53 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:18.326855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499cae1-6162-4eb4-986a-3d2451693b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:18.327135 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499cae1-6162-4eb4-986a-3d2451693b53 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:18.327870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499cae1-6162-4eb4-986a-3d2451693b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:18.329997 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499cae1-6162-4eb4-986a-3d2451693b53 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:18.692872 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1286/2572] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:20:18 2022] POST /identity/v3/users => generated 294 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:20:18.695932 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7daae11-9f91-477b-b422-82f7a2b7cc4d admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:18.707317 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7daae11-9f91-477b-b422-82f7a2b7cc4d admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:18.712941 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1518205-ac75-42cd-8246-80c3e956d796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:18.713686 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1518205-ac75-42cd-8246-80c3e956d796 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:18.713998 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1518205-ac75-42cd-8246-80c3e956d796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:18.714348 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1518205-ac75-42cd-8246-80c3e956d796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:18.715056 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1518205-ac75-42cd-8246-80c3e956d796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:18.716966 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1518205-ac75-42cd-8246-80c3e956d796 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:18.731230 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1287/2573] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:20:18.737245 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499cae1-6162-4eb4-986a-3d2451693b53 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:18.753474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499cae1-6162-4eb4-986a-3d2451693b53 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:18.756619 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854a4b4-89ed-4373-a7b1-1c8357f89638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:18.765682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854a4b4-89ed-4373-a7b1-1c8357f89638 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:18.765682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854a4b4-89ed-4373-a7b1-1c8357f89638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:18.765682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854a4b4-89ed-4373-a7b1-1c8357f89638 admin admin] PATH_INFO: `/v3/projects/f71747b73393418eaaad8004c22effc8/users/535c426b63ff4b508d44d8267267bdc9/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:18.778467 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854a4b4-89ed-4373-a7b1-1c8357f89638 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f71747b73393418eaaad8004c22effc8, user_id=535c426b63ff4b508d44d8267267bdc9, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:18.781332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854a4b4-89ed-4373-a7b1-1c8357f89638 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:18.977207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1287/2574] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:20:18 2022] PUT /identity/v3/projects/f71747b73393418eaaad8004c22effc8/users/535c426b63ff4b508d44d8267267bdc9/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:18.981842 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1518205-ac75-42cd-8246-80c3e956d796 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:18.993994 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1518205-ac75-42cd-8246-80c3e956d796 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:19.066879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:19.073380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab807f87-b44a-4d61-9338-2da3e153cfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:19.074123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab807f87-b44a-4d61-9338-2da3e153cfcd admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:19.074416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab807f87-b44a-4d61-9338-2da3e153cfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:19.074730 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab807f87-b44a-4d61-9338-2da3e153cfcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:19.075471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab807f87-b44a-4d61-9338-2da3e153cfcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:19.077203 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab807f87-b44a-4d61-9338-2da3e153cfcd admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:19.088690 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1288/2575] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:20:19.093115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854a4b4-89ed-4373-a7b1-1c8357f89638 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:19.106816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854a4b4-89ed-4373-a7b1-1c8357f89638 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:19.111792 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b7e71-cafd-4aed-954c-0b705536fba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:19.112802 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b7e71-cafd-4aed-954c-0b705536fba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:19.113167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b7e71-cafd-4aed-954c-0b705536fba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:19.113523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b7e71-cafd-4aed-954c-0b705536fba7 admin admin] PATH_INFO: `/v3/projects/f71747b73393418eaaad8004c22effc8/users/535c426b63ff4b508d44d8267267bdc9/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:19.115727 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b7e71-cafd-4aed-954c-0b705536fba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f71747b73393418eaaad8004c22effc8, user_id=535c426b63ff4b508d44d8267267bdc9, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:19.119227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b7e71-cafd-4aed-954c-0b705536fba7 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:19.131194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1288/2576] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:20:19 2022] PUT /identity/v3/projects/f71747b73393418eaaad8004c22effc8/users/535c426b63ff4b508d44d8267267bdc9/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:19.143234 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e776c06-33f5-4008-b5e5-b986f911a553 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:19.143513 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e776c06-33f5-4008-b5e5-b986f911a553 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:19.143817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e776c06-33f5-4008-b5e5-b986f911a553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:19.184933 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-6e776c06-33f5-4008-b5e5-b986f911a553 None None] MFA Rules not processed for user `535c426b63ff4b508d44d8267267bdc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:19.186153 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-6e776c06-33f5-4008-b5e5-b986f911a553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:19.285827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1289/2577] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:19.294022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b7e71-cafd-4aed-954c-0b705536fba7 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:19.301857 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a64b7e71-cafd-4aed-954c-0b705536fba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:19.419744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b7e71-cafd-4aed-954c-0b705536fba7 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:19.425634 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a5374-0c3b-4401-af60-e16e87d206d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:19.426386 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a5374-0c3b-4401-af60-e16e87d206d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:19.426718 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a5374-0c3b-4401-af60-e16e87d206d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:19.427004 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a5374-0c3b-4401-af60-e16e87d206d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:19.522173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a5374-0c3b-4401-af60-e16e87d206d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:19.525554 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a5374-0c3b-4401-af60-e16e87d206d1 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:19.537869 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1289/2578] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:20:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:20:22.941242 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e776c06-33f5-4008-b5e5-b986f911a553 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:23.016976 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e776c06-33f5-4008-b5e5-b986f911a553 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:23.021884 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf3f51-f548-49f2-95d9-3ee47dfdd424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:23.023049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf3f51-f548-49f2-95d9-3ee47dfdd424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:23.023542 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf3f51-f548-49f2-95d9-3ee47dfdd424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:23.023933 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf3f51-f548-49f2-95d9-3ee47dfdd424 admin admin] PATH_INFO: `/v3/users/f9cb2fa45c0e4c5abccb89ea88684713` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:23.025295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf3f51-f548-49f2-95d9-3ee47dfdd424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9cb2fa45c0e4c5abccb89ea88684713)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:23.027440 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf3f51-f548-49f2-95d9-3ee47dfdd424 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:23.337802 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-fedf3f51-f548-49f2-95d9-3ee47dfdd424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9cb2fa45c0e4c5abccb89ea88684713'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:23.355513 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-fedf3f51-f548-49f2-95d9-3ee47dfdd424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9cb2fa45c0e4c5abccb89ea88684713'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:23.356830 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-fedf3f51-f548-49f2-95d9-3ee47dfdd424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9cb2fa45c0e4c5abccb89ea88684713'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:23.389651 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-fedf3f51-f548-49f2-95d9-3ee47dfdd424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9cb2fa45c0e4c5abccb89ea88684713'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:23.467928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-fedf3f51-f548-49f2-95d9-3ee47dfdd424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9cb2fa45c0e4c5abccb89ea88684713'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:23.469167 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:23.476026 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1290/2579] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:20:22 2022] DELETE /identity/v3/users/f9cb2fa45c0e4c5abccb89ea88684713 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:23.480918 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f79f0-c257-43d7-8145-f671ea79ab96 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:23.481603 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f79f0-c257-43d7-8145-f671ea79ab96 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:23.481603 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f79f0-c257-43d7-8145-f671ea79ab96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:23.502189 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-850f79f0-c257-43d7-8145-f671ea79ab96 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:23.503604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-850f79f0-c257-43d7-8145-f671ea79ab96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:23.584399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:23.609013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1290/2580] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:23.618927 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf3f51-f548-49f2-95d9-3ee47dfdd424 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:23.620518 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-fedf3f51-f548-49f2-95d9-3ee47dfdd424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:23.704022 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf3f51-f548-49f2-95d9-3ee47dfdd424 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:23.707954 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a71ce2-6490-4a9a-8cb7-bb20a3a910ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:23.708722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a71ce2-6490-4a9a-8cb7-bb20a3a910ac service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:23.709075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a71ce2-6490-4a9a-8cb7-bb20a3a910ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:23.709550 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a71ce2-6490-4a9a-8cb7-bb20a3a910ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:23.724892 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a71ce2-6490-4a9a-8cb7-bb20a3a910ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:23.726561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a71ce2-6490-4a9a-8cb7-bb20a3a910ac service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:23.741864 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1291/2581] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:20:24.379988 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f79f0-c257-43d7-8145-f671ea79ab96 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:24.382232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-850f79f0-c257-43d7-8145-f671ea79ab96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:24.396892 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f79f0-c257-43d7-8145-f671ea79ab96 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:24.405504 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ef180-7fb0-4ad8-89e2-a22a094f577f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:24.406393 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ef180-7fb0-4ad8-89e2-a22a094f577f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:24.406704 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ef180-7fb0-4ad8-89e2-a22a094f577f admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:24.406983 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ef180-7fb0-4ad8-89e2-a22a094f577f admin admin] PATH_INFO: `/v3/projects/20b8703a2b544d6e9504f8fbd7574bff` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:24.408305 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ef180-7fb0-4ad8-89e2-a22a094f577f admin admin] RBAC: Authorizing `identity:delete_project(project_id=20b8703a2b544d6e9504f8fbd7574bff)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:24.410253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ef180-7fb0-4ad8-89e2-a22a094f577f admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:24.542843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-442ef180-7fb0-4ad8-89e2-a22a094f577f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20b8703a2b544d6e9504f8fbd7574bff', 'request_id': 'req-442ef180-7fb0-4ad8-89e2-a22a094f577f'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:24.549273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-442ef180-7fb0-4ad8-89e2-a22a094f577f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20b8703a2b544d6e9504f8fbd7574bff', 'request_id': 'req-442ef180-7fb0-4ad8-89e2-a22a094f577f'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:24.556047 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-442ef180-7fb0-4ad8-89e2-a22a094f577f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20b8703a2b544d6e9504f8fbd7574bff', 'request_id': 'req-442ef180-7fb0-4ad8-89e2-a22a094f577f'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:24.650649 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-442ef180-7fb0-4ad8-89e2-a22a094f577f admin admin] The token cache is being invalidate because project 20b8703a2b544d6e9504f8fbd7574bff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:20:24.651148 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-442ef180-7fb0-4ad8-89e2-a22a094f577f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:24.651823 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-442ef180-7fb0-4ad8-89e2-a22a094f577f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:24.653569 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1291/2582] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:20:24 2022] DELETE /identity/v3/projects/20b8703a2b544d6e9504f8fbd7574bff => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:24.664393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a71ce2-6490-4a9a-8cb7-bb20a3a910ac service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:24.675371 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a71ce2-6490-4a9a-8cb7-bb20a3a910ac service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:24.751911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6bdd8-3343-47d4-a08a-45af4b078a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:24.752960 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6bdd8-3343-47d4-a08a-45af4b078a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:24.753249 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6bdd8-3343-47d4-a08a-45af4b078a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:24.753539 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6bdd8-3343-47d4-a08a-45af4b078a37 admin admin] PATH_INFO: `/v3/users/535c426b63ff4b508d44d8267267bdc9` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:24.754807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6bdd8-3343-47d4-a08a-45af4b078a37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=535c426b63ff4b508d44d8267267bdc9)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:24.757394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6bdd8-3343-47d4-a08a-45af4b078a37 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:25.003110 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-40c6bdd8-3343-47d4-a08a-45af4b078a37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '535c426b63ff4b508d44d8267267bdc9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:25.014876 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-40c6bdd8-3343-47d4-a08a-45af4b078a37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '535c426b63ff4b508d44d8267267bdc9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:25.015775 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-40c6bdd8-3343-47d4-a08a-45af4b078a37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '535c426b63ff4b508d44d8267267bdc9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:25.031811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-40c6bdd8-3343-47d4-a08a-45af4b078a37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '535c426b63ff4b508d44d8267267bdc9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:25.161391 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-40c6bdd8-3343-47d4-a08a-45af4b078a37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '535c426b63ff4b508d44d8267267bdc9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:25.170665 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1292/2583] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:20:24 2022] DELETE /identity/v3/users/535c426b63ff4b508d44d8267267bdc9 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:25.724337 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ef180-7fb0-4ad8-89e2-a22a094f577f admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:25.727309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-442ef180-7fb0-4ad8-89e2-a22a094f577f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:25.757907 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ef180-7fb0-4ad8-89e2-a22a094f577f admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:25.839211 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84c84b-b335-4cc2-916c-21a78d47b8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:25.840115 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84c84b-b335-4cc2-916c-21a78d47b8ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:25.840390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84c84b-b335-4cc2-916c-21a78d47b8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:25.840861 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84c84b-b335-4cc2-916c-21a78d47b8ac admin admin] PATH_INFO: `/v3/projects/f71747b73393418eaaad8004c22effc8` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:25.842119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84c84b-b335-4cc2-916c-21a78d47b8ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=f71747b73393418eaaad8004c22effc8)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:25.844102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84c84b-b335-4cc2-916c-21a78d47b8ac admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:26.008383 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cd84c84b-b335-4cc2-916c-21a78d47b8ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f71747b73393418eaaad8004c22effc8', 'request_id': 'req-cd84c84b-b335-4cc2-916c-21a78d47b8ac'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:26.015213 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cd84c84b-b335-4cc2-916c-21a78d47b8ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f71747b73393418eaaad8004c22effc8', 'request_id': 'req-cd84c84b-b335-4cc2-916c-21a78d47b8ac'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:26.022065 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cd84c84b-b335-4cc2-916c-21a78d47b8ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f71747b73393418eaaad8004c22effc8', 'request_id': 'req-cd84c84b-b335-4cc2-916c-21a78d47b8ac'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:26.097220 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cd84c84b-b335-4cc2-916c-21a78d47b8ac admin admin] The token cache is being invalidate because project f71747b73393418eaaad8004c22effc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:20:26.097626 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cd84c84b-b335-4cc2-916c-21a78d47b8ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:26.098436 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cd84c84b-b335-4cc2-916c-21a78d47b8ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:26.100218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1292/2584] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:20:25 2022] DELETE /identity/v3/projects/f71747b73393418eaaad8004c22effc8 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:26.105673 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8bb21-a085-41e0-83f6-2b451b23ae92 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:26.106220 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8bb21-a085-41e0-83f6-2b451b23ae92 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:26.106220 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8bb21-a085-41e0-83f6-2b451b23ae92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:26.130826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-e0b8bb21-a085-41e0-83f6-2b451b23ae92 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:26.132891 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e0b8bb21-a085-41e0-83f6-2b451b23ae92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:26.147286 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:26.222879 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1293/2585] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:26.230773 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161131fa-814d-40b6-b0e9-2e560a6fa241 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:26.231166 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161131fa-814d-40b6-b0e9-2e560a6fa241 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:26.231524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161131fa-814d-40b6-b0e9-2e560a6fa241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:26.246631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:26.253518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-161131fa-814d-40b6-b0e9-2e560a6fa241 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:26.254809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-161131fa-814d-40b6-b0e9-2e560a6fa241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:26.268647 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1293/2586] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:26.271104 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8bb21-a085-41e0-83f6-2b451b23ae92 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:26.283050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8bb21-a085-41e0-83f6-2b451b23ae92 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:26.288099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaaadd2-9f5c-472b-b253-f65d638f4ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:26.292772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaadd2-9f5c-472b-b253-f65d638f4ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:26.292772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaadd2-9f5c-472b-b253-f65d638f4ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:26.292772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaadd2-9f5c-472b-b253-f65d638f4ab2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:26.292772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaaadd2-9f5c-472b-b253-f65d638f4ab2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:20:26.292772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaaadd2-9f5c-472b-b253-f65d638f4ab2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:26.292772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaaadd2-9f5c-472b-b253-f65d638f4ab2 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:26.305770 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1294/2587] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 15:20:26.310662 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161131fa-814d-40b6-b0e9-2e560a6fa241 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:26.323593 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161131fa-814d-40b6-b0e9-2e560a6fa241 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:26.328954 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4e0bf-570e-45e8-95af-a82a6be45da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:26.329624 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4e0bf-570e-45e8-95af-a82a6be45da8 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:26.329909 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4e0bf-570e-45e8-95af-a82a6be45da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:26.330187 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4e0bf-570e-45e8-95af-a82a6be45da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:26.330991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f4e0bf-570e-45e8-95af-a82a6be45da8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:26.332828 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f4e0bf-570e-45e8-95af-a82a6be45da8 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:26.487510 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1294/2588] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:20:26 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:20:26.492586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaaadd2-9f5c-472b-b253-f65d638f4ab2 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:26.502981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaaadd2-9f5c-472b-b253-f65d638f4ab2 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:26.584286 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcccb5-9e88-42e5-9d5d-8d5f25469fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:26.585289 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcccb5-9e88-42e5-9d5d-8d5f25469fbb admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:26.585581 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcccb5-9e88-42e5-9d5d-8d5f25469fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:26.585922 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcccb5-9e88-42e5-9d5d-8d5f25469fbb admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:26.586889 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bcccb5-9e88-42e5-9d5d-8d5f25469fbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:26.588951 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bcccb5-9e88-42e5-9d5d-8d5f25469fbb admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:26.652969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b67c56-e3e6-4a47-954e-6cae731fe78d None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:26.653248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b67c56-e3e6-4a47-954e-6cae731fe78d None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:26.653574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b67c56-e3e6-4a47-954e-6cae731fe78d None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:26.655234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1295/2589] 10.222.0.83 () {54 vars in 921 bytes} [Mon Jul 18 15:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 15:20:26.660873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfbd4d-4f0d-4e56-b8d9-81d4d284a33f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:26.661164 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfbd4d-4f0d-4e56-b8d9-81d4d284a33f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:26.661474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfbd4d-4f0d-4e56-b8d9-81d4d284a33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:26.681500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-0cbfbd4d-4f0d-4e56-b8d9-81d4d284a33f None None] MFA Rules not processed for user `fd86979e0f0d4a4e9921ee5704901f74`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:26.682770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0cbfbd4d-4f0d-4e56-b8d9-81d4d284a33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:26.720318 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1295/2590] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:20:26 2022] POST /identity/v3/users => generated 294 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:20:26.721704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcccb5-9e88-42e5-9d5d-8d5f25469fbb admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:26.739618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcccb5-9e88-42e5-9d5d-8d5f25469fbb admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:26.746398 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8873d0e-67e3-44e4-9a81-cf523b2e7ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:26.747291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8873d0e-67e3-44e4-9a81-cf523b2e7ced admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:26.747682 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8873d0e-67e3-44e4-9a81-cf523b2e7ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:26.748103 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8873d0e-67e3-44e4-9a81-cf523b2e7ced admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:26.749093 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8873d0e-67e3-44e4-9a81-cf523b2e7ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:26.750167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1296/2591] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Jul 18 15:20:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:20:26.751584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8873d0e-67e3-44e4-9a81-cf523b2e7ced admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:26.762988 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfbd4d-4f0d-4e56-b8d9-81d4d284a33f None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:26.764493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0cbfbd4d-4f0d-4e56-b8d9-81d4d284a33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:26.764935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1296/2592] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:20:26.768668 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8873d0e-67e3-44e4-9a81-cf523b2e7ced admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:26.776155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfbd4d-4f0d-4e56-b8d9-81d4d284a33f None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:26.780688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cfe21-210f-44ae-a765-820115a2cdb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:26.783980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8873d0e-67e3-44e4-9a81-cf523b2e7ced admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:26.784194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cfe21-210f-44ae-a765-820115a2cdb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:26.784194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cfe21-210f-44ae-a765-820115a2cdb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:26.784194 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cfe21-210f-44ae-a765-820115a2cdb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:26.786450 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d9c36-bf79-4f53-9f3d-c396288802d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:26.787402 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d9c36-bf79-4f53-9f3d-c396288802d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:26.787840 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d9c36-bf79-4f53-9f3d-c396288802d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:26.789013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d9c36-bf79-4f53-9f3d-c396288802d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:26.790262 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d9c36-bf79-4f53-9f3d-c396288802d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:26.792275 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d9c36-bf79-4f53-9f3d-c396288802d2 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:26.792576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cfe21-210f-44ae-a765-820115a2cdb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:26.794246 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cfe21-210f-44ae-a765-820115a2cdb7 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:26.804085 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1297/2593] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:20:26.805290 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1297/2594] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:20:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:20:26.815398 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d9c36-bf79-4f53-9f3d-c396288802d2 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:26.825899 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d9c36-bf79-4f53-9f3d-c396288802d2 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:26.830877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe8159-863e-41e5-adfb-8faccd01f475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:26.831807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe8159-863e-41e5-adfb-8faccd01f475 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:26.832096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe8159-863e-41e5-adfb-8faccd01f475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:26.832380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe8159-863e-41e5-adfb-8faccd01f475 admin admin] PATH_INFO: `/v3/projects/9ca0b1aa3e94401da2f50d761d6dc31c/users/e67a931c95d9463faacf74f65761cf7a/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:26.852248 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe8159-863e-41e5-adfb-8faccd01f475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ca0b1aa3e94401da2f50d761d6dc31c, user_id=e67a931c95d9463faacf74f65761cf7a, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:26.854782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe8159-863e-41e5-adfb-8faccd01f475 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:26.922887 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051f0d6-3396-4c89-bea6-04a9c0e1b5a6 None None] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:26.923248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051f0d6-3396-4c89-bea6-04a9c0e1b5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:26.923519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051f0d6-3396-4c89-bea6-04a9c0e1b5a6 None None] PATH_INFO: `/` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:26.924928 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1298/2595] 10.222.0.83 () {54 vars in 921 bytes} [Mon Jul 18 15:20:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 15:20:26.938986 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dcd48-3b81-4e60-8435-8555030eea92 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:26.939287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dcd48-3b81-4e60-8435-8555030eea92 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:26.939573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dcd48-3b81-4e60-8435-8555030eea92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:26.964296 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-f95dcd48-3b81-4e60-8435-8555030eea92 None None] MFA Rules not processed for user `7e1355d18fb049468651b3675c217d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:26.967332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-f95dcd48-3b81-4e60-8435-8555030eea92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:26.977451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1298/2596] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:20:26 2022] PUT /identity/v3/projects/9ca0b1aa3e94401da2f50d761d6dc31c/users/e67a931c95d9463faacf74f65761cf7a/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:26.988991 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b55538-ae09-4bff-8219-b32baab40a80 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:26.989200 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b55538-ae09-4bff-8219-b32baab40a80 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:26.989466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b55538-ae09-4bff-8219-b32baab40a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:27.030915 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-c3b55538-ae09-4bff-8219-b32baab40a80 None None] MFA Rules not processed for user `e67a931c95d9463faacf74f65761cf7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:27.032232 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c3b55538-ae09-4bff-8219-b32baab40a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:27.071424 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1299/2597] 10.222.0.83 () {58 vars in 1018 bytes} [Mon Jul 18 15:20:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:27.084648 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dcd48-3b81-4e60-8435-8555030eea92 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:27.112252 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1299/2598] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:27.123751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b55538-ae09-4bff-8219-b32baab40a80 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:27.125565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c3b55538-ae09-4bff-8219-b32baab40a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:27.175786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dcd48-3b81-4e60-8435-8555030eea92 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:27.182276 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a09a77-3d19-4674-8ba9-a823e41e1065 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:27.183006 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a09a77-3d19-4674-8ba9-a823e41e1065 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:27.183299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a09a77-3d19-4674-8ba9-a823e41e1065 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:27.183581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a09a77-3d19-4674-8ba9-a823e41e1065 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:27.195037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a09a77-3d19-4674-8ba9-a823e41e1065 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:27.197172 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a09a77-3d19-4674-8ba9-a823e41e1065 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:27.200665 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b55538-ae09-4bff-8219-b32baab40a80 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:27.205913 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6dfdc-b3f6-45f5-b18b-a53a7afb7440 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:27.206881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6dfdc-b3f6-45f5-b18b-a53a7afb7440 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:27.207299 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6dfdc-b3f6-45f5-b18b-a53a7afb7440 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:27.207678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6dfdc-b3f6-45f5-b18b-a53a7afb7440 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:27.209242 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1300/2599] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:20:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:20:27.218742 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6dfdc-b3f6-45f5-b18b-a53a7afb7440 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:27.220238 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6dfdc-b3f6-45f5-b18b-a53a7afb7440 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:27.231333 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1300/2600] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:20:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:20:27.305269 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:37.453407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=95,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:38.025923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8134d-e68b-41da-8933-9733e09692d2 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:38.026218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8134d-e68b-41da-8933-9733e09692d2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:38.026501 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8134d-e68b-41da-8933-9733e09692d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:38.046700 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-54d8134d-e68b-41da-8933-9733e09692d2 None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:38.047956 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-54d8134d-e68b-41da-8933-9733e09692d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:38.109831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1301/2601] 10.222.0.83 () {58 vars in 1009 bytes} [Mon Jul 18 15:20:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:20:38.122719 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6dfdc-b3f6-45f5-b18b-a53a7afb7440 service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:38.124303 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-32a6dfdc-b3f6-45f5-b18b-a53a7afb7440 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:38.201608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6dfdc-b3f6-45f5-b18b-a53a7afb7440 service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:38.206927 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340feed4-7d69-49c5-b0dd-c78a7f65c9ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:38.209341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340feed4-7d69-49c5-b0dd-c78a7f65c9ba service swift] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:38.209341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340feed4-7d69-49c5-b0dd-c78a7f65c9ba service swift] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:38.209341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340feed4-7d69-49c5-b0dd-c78a7f65c9ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:38.218330 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340feed4-7d69-49c5-b0dd-c78a7f65c9ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:38.220232 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340feed4-7d69-49c5-b0dd-c78a7f65c9ba service swift] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:38.230225 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1301/2602] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:20:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 15:20:41.039824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8134d-e68b-41da-8933-9733e09692d2 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:41.041503 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-54d8134d-e68b-41da-8933-9733e09692d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:41.191327 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8134d-e68b-41da-8933-9733e09692d2 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:41.196849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a191dd9-a7e6-481f-be4f-5124e802bf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:41.198286 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a191dd9-a7e6-481f-be4f-5124e802bf98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:41.198786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a191dd9-a7e6-481f-be4f-5124e802bf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:41.199248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a191dd9-a7e6-481f-be4f-5124e802bf98 admin admin] PATH_INFO: `/v3/users/4746f061cfd148609ffaf568f869d6d8` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:41.200839 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a191dd9-a7e6-481f-be4f-5124e802bf98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4746f061cfd148609ffaf568f869d6d8)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:41.202950 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a191dd9-a7e6-481f-be4f-5124e802bf98 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:41.417086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9a191dd9-a7e6-481f-be4f-5124e802bf98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4746f061cfd148609ffaf568f869d6d8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:41.418216 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9a191dd9-a7e6-481f-be4f-5124e802bf98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4746f061cfd148609ffaf568f869d6d8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:41.419367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9a191dd9-a7e6-481f-be4f-5124e802bf98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4746f061cfd148609ffaf568f869d6d8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:41.506847 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9a191dd9-a7e6-481f-be4f-5124e802bf98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4746f061cfd148609ffaf568f869d6d8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:41.527044 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9a191dd9-a7e6-481f-be4f-5124e802bf98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4746f061cfd148609ffaf568f869d6d8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:41.546343 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1302/2603] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:20:41 2022] DELETE /identity/v3/users/4746f061cfd148609ffaf568f869d6d8 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:41.984260 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340feed4-7d69-49c5-b0dd-c78a7f65c9ba service swift] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:42.004416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340feed4-7d69-49c5-b0dd-c78a7f65c9ba service swift] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:42.091782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9178fa8-8a57-429a-b4db-214e7b867e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:42.092756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9178fa8-8a57-429a-b4db-214e7b867e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:42.093072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9178fa8-8a57-429a-b4db-214e7b867e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:42.093384 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9178fa8-8a57-429a-b4db-214e7b867e8d admin admin] PATH_INFO: `/v3/projects/fcd59138153e461bb239770cc1bbc1f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:42.094744 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9178fa8-8a57-429a-b4db-214e7b867e8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcd59138153e461bb239770cc1bbc1f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:42.097001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9178fa8-8a57-429a-b4db-214e7b867e8d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:42.114882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:42.233940 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c9178fa8-8a57-429a-b4db-214e7b867e8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcd59138153e461bb239770cc1bbc1f1', 'request_id': 'req-c9178fa8-8a57-429a-b4db-214e7b867e8d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:42.240837 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c9178fa8-8a57-429a-b4db-214e7b867e8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcd59138153e461bb239770cc1bbc1f1', 'request_id': 'req-c9178fa8-8a57-429a-b4db-214e7b867e8d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:42.300558 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c9178fa8-8a57-429a-b4db-214e7b867e8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcd59138153e461bb239770cc1bbc1f1', 'request_id': 'req-c9178fa8-8a57-429a-b4db-214e7b867e8d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:42.314591 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c9178fa8-8a57-429a-b4db-214e7b867e8d admin admin] The token cache is being invalidate because project fcd59138153e461bb239770cc1bbc1f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:20:42.315022 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c9178fa8-8a57-429a-b4db-214e7b867e8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:42.316028 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c9178fa8-8a57-429a-b4db-214e7b867e8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:42.317759 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1302/2604] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:20:41 2022] DELETE /identity/v3/projects/fcd59138153e461bb239770cc1bbc1f1 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:51.000441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a191dd9-a7e6-481f-be4f-5124e802bf98 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:51.002722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9a191dd9-a7e6-481f-be4f-5124e802bf98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:51.030183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a191dd9-a7e6-481f-be4f-5124e802bf98 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:51.056558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:51.098800 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7da0ff-ab8c-4bf4-bb9e-14f1ec24be7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:51.099723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7da0ff-ab8c-4bf4-bb9e-14f1ec24be7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:51.100007 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7da0ff-ab8c-4bf4-bb9e-14f1ec24be7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:51.100288 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7da0ff-ab8c-4bf4-bb9e-14f1ec24be7a admin admin] PATH_INFO: `/v3/users/e67a931c95d9463faacf74f65761cf7a` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:51.102057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7da0ff-ab8c-4bf4-bb9e-14f1ec24be7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e67a931c95d9463faacf74f65761cf7a)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:51.104423 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7da0ff-ab8c-4bf4-bb9e-14f1ec24be7a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:51.395207 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ab7da0ff-ab8c-4bf4-bb9e-14f1ec24be7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e67a931c95d9463faacf74f65761cf7a'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:51.396068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ab7da0ff-ab8c-4bf4-bb9e-14f1ec24be7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e67a931c95d9463faacf74f65761cf7a'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:51.396941 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ab7da0ff-ab8c-4bf4-bb9e-14f1ec24be7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e67a931c95d9463faacf74f65761cf7a'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:51.500522 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ab7da0ff-ab8c-4bf4-bb9e-14f1ec24be7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e67a931c95d9463faacf74f65761cf7a'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:51.525952 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ab7da0ff-ab8c-4bf4-bb9e-14f1ec24be7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e67a931c95d9463faacf74f65761cf7a'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:51.554335 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1303/2605] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:20:50 2022] DELETE /identity/v3/users/e67a931c95d9463faacf74f65761cf7a => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:51.559505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad2680-4ec6-4668-abf4-dbf3955027d1 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:51.564204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad2680-4ec6-4668-abf4-dbf3955027d1 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:51.564204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad2680-4ec6-4668-abf4-dbf3955027d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:51.597721 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-d4ad2680-4ec6-4668-abf4-dbf3955027d1 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:51.599691 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d4ad2680-4ec6-4668-abf4-dbf3955027d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:51.679974 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1303/2606] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:51.690592 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7da0ff-ab8c-4bf4-bb9e-14f1ec24be7a admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:51.692263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ab7da0ff-ab8c-4bf4-bb9e-14f1ec24be7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:51.757296 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7da0ff-ab8c-4bf4-bb9e-14f1ec24be7a admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:51.761657 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ba9f7-ac1b-4ec9-958a-46d48e9fdee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:51.762311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ba9f7-ac1b-4ec9-958a-46d48e9fdee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:51.762623 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ba9f7-ac1b-4ec9-958a-46d48e9fdee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:51.762911 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ba9f7-ac1b-4ec9-958a-46d48e9fdee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:51.772441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ba9f7-ac1b-4ec9-958a-46d48e9fdee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:51.776689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ba9f7-ac1b-4ec9-958a-46d48e9fdee5 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:51.784611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1304/2607] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:20:52.284406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad2680-4ec6-4668-abf4-dbf3955027d1 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:52.285950 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d4ad2680-4ec6-4668-abf4-dbf3955027d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:52.300235 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad2680-4ec6-4668-abf4-dbf3955027d1 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:52.304918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3582d-daa7-48e9-b410-a271d2bcff0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:52.305787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3582d-daa7-48e9-b410-a271d2bcff0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:52.306072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3582d-daa7-48e9-b410-a271d2bcff0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:52.306350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3582d-daa7-48e9-b410-a271d2bcff0d admin admin] PATH_INFO: `/v3/projects/9ca0b1aa3e94401da2f50d761d6dc31c` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:52.307588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3582d-daa7-48e9-b410-a271d2bcff0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ca0b1aa3e94401da2f50d761d6dc31c)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:52.309528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3582d-daa7-48e9-b410-a271d2bcff0d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:52.467717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0cf3582d-daa7-48e9-b410-a271d2bcff0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ca0b1aa3e94401da2f50d761d6dc31c', 'request_id': 'req-0cf3582d-daa7-48e9-b410-a271d2bcff0d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:52.475280 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0cf3582d-daa7-48e9-b410-a271d2bcff0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ca0b1aa3e94401da2f50d761d6dc31c', 'request_id': 'req-0cf3582d-daa7-48e9-b410-a271d2bcff0d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:52.584115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0cf3582d-daa7-48e9-b410-a271d2bcff0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ca0b1aa3e94401da2f50d761d6dc31c', 'request_id': 'req-0cf3582d-daa7-48e9-b410-a271d2bcff0d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:52.600265 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0cf3582d-daa7-48e9-b410-a271d2bcff0d admin admin] The token cache is being invalidate because project 9ca0b1aa3e94401da2f50d761d6dc31c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:20:52.600265 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0cf3582d-daa7-48e9-b410-a271d2bcff0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:52.600265 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-0cf3582d-daa7-48e9-b410-a271d2bcff0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:20:52.601493 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1304/2608] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:20:52 2022] DELETE /identity/v3/projects/9ca0b1aa3e94401da2f50d761d6dc31c => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:20:56.538765 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a08f-952e-4071-9d23-6c0f2112b2aa None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:56.539798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a08f-952e-4071-9d23-6c0f2112b2aa None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:56.539986 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a08f-952e-4071-9d23-6c0f2112b2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:56.564798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-7a90a08f-952e-4071-9d23-6c0f2112b2aa None None] MFA Rules not processed for user `692fef632836474da09f0e8a009ffa4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:20:56.566407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7a90a08f-952e-4071-9d23-6c0f2112b2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:56.624143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:56.640390 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1305/2609] 10.222.0.83 () {58 vars in 967 bytes} [Mon Jul 18 15:20:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:20:56.653636 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3582d-daa7-48e9-b410-a271d2bcff0d admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:20:56.655154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-0cf3582d-daa7-48e9-b410-a271d2bcff0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:20:56.687672 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:20:56.729090 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3582d-daa7-48e9-b410-a271d2bcff0d admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:20:56.734399 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c526f7-153a-4645-9186-9b039719e3d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88dfe620ff04ffe88088a8a23c7d6e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:20:56.735095 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c526f7-153a-4645-9186-9b039719e3d4 service swift] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:20:56.735415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c526f7-153a-4645-9186-9b039719e3d4 service swift] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:20:56.735701 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c526f7-153a-4645-9186-9b039719e3d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:20:56.744752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c526f7-153a-4645-9186-9b039719e3d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:20:56.746431 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c526f7-153a-4645-9186-9b039719e3d4 service swift] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:20:56.755257 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1305/2610] 10.222.0.83 () {60 vars in 1384 bytes} [Mon Jul 18 15:20:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 15:21:03.020479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90a08f-952e-4071-9d23-6c0f2112b2aa None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:03.021966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7a90a08f-952e-4071-9d23-6c0f2112b2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:03.081849 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90a08f-952e-4071-9d23-6c0f2112b2aa None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:03.085502 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99b118-c7e2-4f4b-a7a2-6c9c55bfd165 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:03.087107 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99b118-c7e2-4f4b-a7a2-6c9c55bfd165 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:03.087107 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99b118-c7e2-4f4b-a7a2-6c9c55bfd165 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:03.087367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99b118-c7e2-4f4b-a7a2-6c9c55bfd165 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:03.089256 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ec99b118-c7e2-4f4b-a7a2-6c9c55bfd165 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:03.162404 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99b118-c7e2-4f4b-a7a2-6c9c55bfd165 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:03.163886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99b118-c7e2-4f4b-a7a2-6c9c55bfd165 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:03.174444 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1306/2611] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:21:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:21:06.905517 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=20 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:21:07.544330 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a252051-39ee-4c98-9056-1ab03c9b9ca3 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:07.544620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a252051-39ee-4c98-9056-1ab03c9b9ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:07.544902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a252051-39ee-4c98-9056-1ab03c9b9ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:07.577196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-3a252051-39ee-4c98-9056-1ab03c9b9ca3 None None] MFA Rules not processed for user `6bfa2a00688d4eab993059ff176d8920`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:21:07.578494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3a252051-39ee-4c98-9056-1ab03c9b9ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:07.649238 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99b118-c7e2-4f4b-a7a2-6c9c55bfd165 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:07.650359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ec99b118-c7e2-4f4b-a7a2-6c9c55bfd165 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:07.656323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1306/2612] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:21:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:21:07.739881 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99b118-c7e2-4f4b-a7a2-6c9c55bfd165 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:07.744921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015feca-a2e3-42bd-af89-9a01ae9ea1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:07.745731 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015feca-a2e3-42bd-af89-9a01ae9ea1b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:07.746016 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015feca-a2e3-42bd-af89-9a01ae9ea1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:07.746296 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015feca-a2e3-42bd-af89-9a01ae9ea1b6 admin admin] PATH_INFO: `/v3/users/22b86eef7820462fa4e6cbb22f30b853` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:07.747545 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015feca-a2e3-42bd-af89-9a01ae9ea1b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22b86eef7820462fa4e6cbb22f30b853)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:07.749516 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015feca-a2e3-42bd-af89-9a01ae9ea1b6 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:07.888755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:21:07.898058 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-1015feca-a2e3-42bd-af89-9a01ae9ea1b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22b86eef7820462fa4e6cbb22f30b853'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:07.899348 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-1015feca-a2e3-42bd-af89-9a01ae9ea1b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22b86eef7820462fa4e6cbb22f30b853'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:07.900026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-1015feca-a2e3-42bd-af89-9a01ae9ea1b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22b86eef7820462fa4e6cbb22f30b853'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:07.968574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-1015feca-a2e3-42bd-af89-9a01ae9ea1b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22b86eef7820462fa4e6cbb22f30b853'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:07.984880 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-1015feca-a2e3-42bd-af89-9a01ae9ea1b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22b86eef7820462fa4e6cbb22f30b853'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:08.002500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1307/2613] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:21:07 2022] DELETE /identity/v3/users/22b86eef7820462fa4e6cbb22f30b853 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:21:08.307704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a252051-39ee-4c98-9056-1ab03c9b9ca3 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:08.319855 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a252051-39ee-4c98-9056-1ab03c9b9ca3 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:08.394441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c5ea9-e4dd-49aa-92b2-4c37940320b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:08.395363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c5ea9-e4dd-49aa-92b2-4c37940320b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:08.395649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c5ea9-e4dd-49aa-92b2-4c37940320b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:08.395929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c5ea9-e4dd-49aa-92b2-4c37940320b5 admin admin] PATH_INFO: `/v3/projects/7cb30e124ce7458f9c29e6c6ae3d95f0` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:08.397148 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c5ea9-e4dd-49aa-92b2-4c37940320b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cb30e124ce7458f9c29e6c6ae3d95f0)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:08.400970 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c5ea9-e4dd-49aa-92b2-4c37940320b5 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:08.599573 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b22c5ea9-e4dd-49aa-92b2-4c37940320b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cb30e124ce7458f9c29e6c6ae3d95f0', 'request_id': 'req-b22c5ea9-e4dd-49aa-92b2-4c37940320b5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:08.606859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b22c5ea9-e4dd-49aa-92b2-4c37940320b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cb30e124ce7458f9c29e6c6ae3d95f0', 'request_id': 'req-b22c5ea9-e4dd-49aa-92b2-4c37940320b5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:08.723143 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b22c5ea9-e4dd-49aa-92b2-4c37940320b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cb30e124ce7458f9c29e6c6ae3d95f0', 'request_id': 'req-b22c5ea9-e4dd-49aa-92b2-4c37940320b5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:08.735894 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b22c5ea9-e4dd-49aa-92b2-4c37940320b5 admin admin] The token cache is being invalidate because project 7cb30e124ce7458f9c29e6c6ae3d95f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:21:08.736176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b22c5ea9-e4dd-49aa-92b2-4c37940320b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:08.736863 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b22c5ea9-e4dd-49aa-92b2-4c37940320b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:08.738670 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1307/2614] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:21:08 2022] DELETE /identity/v3/projects/7cb30e124ce7458f9c29e6c6ae3d95f0 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:21:08.744328 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11f1f9-08a4-4622-97a2-3cdafdc4f1e0 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:08.744948 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11f1f9-08a4-4622-97a2-3cdafdc4f1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:08.745235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11f1f9-08a4-4622-97a2-3cdafdc4f1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:08.813073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-eb11f1f9-08a4-4622-97a2-3cdafdc4f1e0 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:21:08.814257 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-eb11f1f9-08a4-4622-97a2-3cdafdc4f1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:08.897287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1308/2615] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:21:08.909646 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e8638-d7a8-403b-955d-6baf2ef6b5a0 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:08.909912 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e8638-d7a8-403b-955d-6baf2ef6b5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:08.910811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e8638-d7a8-403b-955d-6baf2ef6b5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:08.934466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-d73e8638-d7a8-403b-955d-6baf2ef6b5a0 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:21:08.935826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d73e8638-d7a8-403b-955d-6baf2ef6b5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:08.946838 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1308/2616] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:21:08.951473 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11f1f9-08a4-4622-97a2-3cdafdc4f1e0 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:08.968032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11f1f9-08a4-4622-97a2-3cdafdc4f1e0 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:08.973996 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbca690-2d4d-44c8-be7a-ac88bdeddc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:08.974810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbca690-2d4d-44c8-be7a-ac88bdeddc3d admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:08.975174 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbca690-2d4d-44c8-be7a-ac88bdeddc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:08.975523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbca690-2d4d-44c8-be7a-ac88bdeddc3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:08.976394 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbca690-2d4d-44c8-be7a-ac88bdeddc3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:21:08.977470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbca690-2d4d-44c8-be7a-ac88bdeddc3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:08.979424 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbca690-2d4d-44c8-be7a-ac88bdeddc3d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:08.998912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1309/2617] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:21:09.002669 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e8638-d7a8-403b-955d-6baf2ef6b5a0 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:09.007252 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:21:09.015142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e8638-d7a8-403b-955d-6baf2ef6b5a0 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:09.019669 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419f998-afb7-401d-8d36-fd9262013872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:09.020242 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419f998-afb7-401d-8d36-fd9262013872 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:09.020525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419f998-afb7-401d-8d36-fd9262013872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:09.020801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419f998-afb7-401d-8d36-fd9262013872 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:09.021524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419f998-afb7-401d-8d36-fd9262013872 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:09.024075 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419f998-afb7-401d-8d36-fd9262013872 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:09.135247 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1309/2618] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:21:09 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:21:09.139562 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbca690-2d4d-44c8-be7a-ac88bdeddc3d admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:09.150494 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbca690-2d4d-44c8-be7a-ac88bdeddc3d admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:09.214378 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d187509-3a99-4507-a6e7-3081c84f152a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:09.215178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d187509-3a99-4507-a6e7-3081c84f152a admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:09.215391 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d187509-3a99-4507-a6e7-3081c84f152a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:09.215677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d187509-3a99-4507-a6e7-3081c84f152a admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:09.218392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d187509-3a99-4507-a6e7-3081c84f152a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:09.218392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d187509-3a99-4507-a6e7-3081c84f152a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:09.523802 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:21:09.532050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1310/2619] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:21:09 2022] POST /identity/v3/users => generated 292 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:21:09.537342 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419f998-afb7-401d-8d36-fd9262013872 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:09.549109 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419f998-afb7-401d-8d36-fd9262013872 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:09.554286 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca51c1-11bd-43d2-9970-a35b848c3917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:09.555266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca51c1-11bd-43d2-9970-a35b848c3917 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:09.556007 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca51c1-11bd-43d2-9970-a35b848c3917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:09.556007 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca51c1-11bd-43d2-9970-a35b848c3917 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:09.557174 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca51c1-11bd-43d2-9970-a35b848c3917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:09.558472 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca51c1-11bd-43d2-9970-a35b848c3917 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:09.569704 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1310/2620] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:21:09.574330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d187509-3a99-4507-a6e7-3081c84f152a admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:09.584874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d187509-3a99-4507-a6e7-3081c84f152a admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:09.591076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543438f7-0f9d-4505-97b1-89311738b6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:09.591765 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543438f7-0f9d-4505-97b1-89311738b6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:09.592057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543438f7-0f9d-4505-97b1-89311738b6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:09.592372 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543438f7-0f9d-4505-97b1-89311738b6c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:09.593118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543438f7-0f9d-4505-97b1-89311738b6c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:09.594769 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543438f7-0f9d-4505-97b1-89311738b6c7 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:09.607525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1311/2621] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:21:09.612048 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca51c1-11bd-43d2-9970-a35b848c3917 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:09.623267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca51c1-11bd-43d2-9970-a35b848c3917 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:09.628588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f64f00-fe05-44a2-87db-d8b72b2830b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:09.629278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f64f00-fe05-44a2-87db-d8b72b2830b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:09.629570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f64f00-fe05-44a2-87db-d8b72b2830b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:09.629855 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f64f00-fe05-44a2-87db-d8b72b2830b3 admin admin] PATH_INFO: `/v3/projects/558c29b107e14e3f9b2aa041a388dfa4/users/5a9a938a9006499a9426af14cb182ecc/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:09.647029 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f64f00-fe05-44a2-87db-d8b72b2830b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=558c29b107e14e3f9b2aa041a388dfa4, user_id=5a9a938a9006499a9426af14cb182ecc, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:09.650360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f64f00-fe05-44a2-87db-d8b72b2830b3 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:09.733399 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1311/2622] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:21:09 2022] PUT /identity/v3/projects/558c29b107e14e3f9b2aa041a388dfa4/users/5a9a938a9006499a9426af14cb182ecc/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:21:09.738873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fc974-6746-4c71-88ba-9f76199dbad0 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:09.739172 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fc974-6746-4c71-88ba-9f76199dbad0 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:09.739490 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fc974-6746-4c71-88ba-9f76199dbad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:09.757637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d14fc974-6746-4c71-88ba-9f76199dbad0 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:21:09.764444 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d14fc974-6746-4c71-88ba-9f76199dbad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:09.822725 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1312/2623] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:21:09.832981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f64f00-fe05-44a2-87db-d8b72b2830b3 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:09.834878 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-87f64f00-fe05-44a2-87db-d8b72b2830b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:09.908535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f64f00-fe05-44a2-87db-d8b72b2830b3 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:09.912349 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842656ba-d367-4e00-b946-63f39ffbc750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:09.913005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842656ba-d367-4e00-b946-63f39ffbc750 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:09.913284 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842656ba-d367-4e00-b946-63f39ffbc750 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:09.913563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842656ba-d367-4e00-b946-63f39ffbc750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:09.926060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842656ba-d367-4e00-b946-63f39ffbc750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:09.927597 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842656ba-d367-4e00-b946-63f39ffbc750 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:09.941909 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1312/2624] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:21:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:21:15.022057 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fc974-6746-4c71-88ba-9f76199dbad0 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:15.024153 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d14fc974-6746-4c71-88ba-9f76199dbad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:15.228932 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fc974-6746-4c71-88ba-9f76199dbad0 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:15.244588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b1c99-2405-4032-ba70-cd195fe19adc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76cc0ae48b5a49c6a7a1c11a08c389e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:15.245707 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b1c99-2405-4032-ba70-cd195fe19adc service placement] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:15.246244 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b1c99-2405-4032-ba70-cd195fe19adc service placement] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:15.247114 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b1c99-2405-4032-ba70-cd195fe19adc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:15.255552 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-865b1c99-2405-4032-ba70-cd195fe19adc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:15.276860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b1c99-2405-4032-ba70-cd195fe19adc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:15.276860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b1c99-2405-4032-ba70-cd195fe19adc service placement] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:15.301645 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1313/2625] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:21:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:21:16.718240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820d526-289e-4d85-bb9a-8cd647e600fa None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:16.718978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820d526-289e-4d85-bb9a-8cd647e600fa None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:16.719261 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820d526-289e-4d85-bb9a-8cd647e600fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:16.750324 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-0820d526-289e-4d85-bb9a-8cd647e600fa None None] MFA Rules not processed for user `5a9a938a9006499a9426af14cb182ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:21:16.751608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-0820d526-289e-4d85-bb9a-8cd647e600fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:16.822143 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1313/2626] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:21:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:21:16.837350 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b1c99-2405-4032-ba70-cd195fe19adc service placement] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:16.839128 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-865b1c99-2405-4032-ba70-cd195fe19adc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:16.903879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:21:16.949215 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b1c99-2405-4032-ba70-cd195fe19adc service placement] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:16.955974 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5949e-57cd-4904-b926-fe7dca8029fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:16.956740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5949e-57cd-4904-b926-fe7dca8029fd service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:16.956937 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5949e-57cd-4904-b926-fe7dca8029fd service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:16.963235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5949e-57cd-4904-b926-fe7dca8029fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:16.966780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5949e-57cd-4904-b926-fe7dca8029fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:16.968451 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5949e-57cd-4904-b926-fe7dca8029fd service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:16.985235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1314/2627] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:21:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:21:20.064570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef5e14-a69a-4b76-9318-2ae0e957ca6b None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:20.065234 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef5e14-a69a-4b76-9318-2ae0e957ca6b None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:20.065234 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef5e14-a69a-4b76-9318-2ae0e957ca6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:20.085897 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-03ef5e14-a69a-4b76-9318-2ae0e957ca6b None None] MFA Rules not processed for user `5a9a938a9006499a9426af14cb182ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:21:20.087574 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-03ef5e14-a69a-4b76-9318-2ae0e957ca6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:20.095082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1314/2628] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:21:26.975505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=20 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:21:30.079839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=91,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:21:41.011868 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5949e-57cd-4904-b926-fe7dca8029fd service nova] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:41.031507 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5949e-57cd-4904-b926-fe7dca8029fd service nova] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:41.035067 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c52ff5-878e-406f-b716-197efb7fc211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:41.035702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c52ff5-878e-406f-b716-197efb7fc211 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:41.035986 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c52ff5-878e-406f-b716-197efb7fc211 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:41.036266 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c52ff5-878e-406f-b716-197efb7fc211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:41.038524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-91c52ff5-878e-406f-b716-197efb7fc211 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:41.050802 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c52ff5-878e-406f-b716-197efb7fc211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:41.052240 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c52ff5-878e-406f-b716-197efb7fc211 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:41.063185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1315/2629] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:21:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:21:44.934561 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef5e14-a69a-4b76-9318-2ae0e957ca6b None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:44.936216 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-03ef5e14-a69a-4b76-9318-2ae0e957ca6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:44.951684 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef5e14-a69a-4b76-9318-2ae0e957ca6b None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:44.956520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e31c23-7402-476e-b59a-8161061844d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:44.957437 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e31c23-7402-476e-b59a-8161061844d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:44.957722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e31c23-7402-476e-b59a-8161061844d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:44.957998 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e31c23-7402-476e-b59a-8161061844d8 admin admin] PATH_INFO: `/v3/users/f3f0c37e426640f9960c1da232605e55` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:44.959254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e31c23-7402-476e-b59a-8161061844d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3f0c37e426640f9960c1da232605e55)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:44.962080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e31c23-7402-476e-b59a-8161061844d8 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:45.211227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f4e31c23-7402-476e-b59a-8161061844d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3f0c37e426640f9960c1da232605e55'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:45.218719 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f4e31c23-7402-476e-b59a-8161061844d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3f0c37e426640f9960c1da232605e55'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:45.219428 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f4e31c23-7402-476e-b59a-8161061844d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3f0c37e426640f9960c1da232605e55'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:45.231984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f4e31c23-7402-476e-b59a-8161061844d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3f0c37e426640f9960c1da232605e55'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:45.351209 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f4e31c23-7402-476e-b59a-8161061844d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3f0c37e426640f9960c1da232605e55'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:45.357835 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1315/2630] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:21:44 2022] DELETE /identity/v3/users/f3f0c37e426640f9960c1da232605e55 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:21:45.615769 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c52ff5-878e-406f-b716-197efb7fc211 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:45.627364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c52ff5-878e-406f-b716-197efb7fc211 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:45.693535 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8486e-22bf-41a5-b238-1d65eeeaf1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:45.694662 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8486e-22bf-41a5-b238-1d65eeeaf1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:45.694967 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8486e-22bf-41a5-b238-1d65eeeaf1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:45.695241 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8486e-22bf-41a5-b238-1d65eeeaf1e5 admin admin] PATH_INFO: `/v3/projects/ea95f701696f4c28b2d4bbef9d0af45d` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:45.696532 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8486e-22bf-41a5-b238-1d65eeeaf1e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea95f701696f4c28b2d4bbef9d0af45d)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:45.698430 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8486e-22bf-41a5-b238-1d65eeeaf1e5 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:45.827319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-faa8486e-22bf-41a5-b238-1d65eeeaf1e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea95f701696f4c28b2d4bbef9d0af45d', 'request_id': 'req-faa8486e-22bf-41a5-b238-1d65eeeaf1e5'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:45.833621 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-faa8486e-22bf-41a5-b238-1d65eeeaf1e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea95f701696f4c28b2d4bbef9d0af45d', 'request_id': 'req-faa8486e-22bf-41a5-b238-1d65eeeaf1e5'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:45.840594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-faa8486e-22bf-41a5-b238-1d65eeeaf1e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea95f701696f4c28b2d4bbef9d0af45d', 'request_id': 'req-faa8486e-22bf-41a5-b238-1d65eeeaf1e5'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:45.904246 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-faa8486e-22bf-41a5-b238-1d65eeeaf1e5 admin admin] The token cache is being invalidate because project ea95f701696f4c28b2d4bbef9d0af45d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:21:45.904552 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-faa8486e-22bf-41a5-b238-1d65eeeaf1e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:45.905182 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-faa8486e-22bf-41a5-b238-1d65eeeaf1e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:21:45.906879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1316/2631] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:21:45 2022] DELETE /identity/v3/projects/ea95f701696f4c28b2d4bbef9d0af45d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:21:45.912190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ed27d-6298-4bcd-900e-e4028dd59118 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:45.912484 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ed27d-6298-4bcd-900e-e4028dd59118 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:45.912779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ed27d-6298-4bcd-900e-e4028dd59118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:45.931886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-4c9ed27d-6298-4bcd-900e-e4028dd59118 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:21:45.933137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4c9ed27d-6298-4bcd-900e-e4028dd59118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:45.994608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1316/2632] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:21:46.002878 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d273b5-407c-408e-9ec5-8f1c9107d33c None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:46.003177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d273b5-407c-408e-9ec5-8f1c9107d33c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:46.003491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d273b5-407c-408e-9ec5-8f1c9107d33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:46.145537 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-e7d273b5-407c-408e-9ec5-8f1c9107d33c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:21:46.146716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e7d273b5-407c-408e-9ec5-8f1c9107d33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:46.156223 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1317/2633] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:21:46.160748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ed27d-6298-4bcd-900e-e4028dd59118 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:46.171861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ed27d-6298-4bcd-900e-e4028dd59118 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:46.176508 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b594e3-e06e-425b-868a-5402372d5042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:46.177169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b594e3-e06e-425b-868a-5402372d5042 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:46.177462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b594e3-e06e-425b-868a-5402372d5042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:46.177748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b594e3-e06e-425b-868a-5402372d5042 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:46.178718 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b594e3-e06e-425b-868a-5402372d5042 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:21:46.179138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b594e3-e06e-425b-868a-5402372d5042 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:46.180758 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b594e3-e06e-425b-868a-5402372d5042 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:46.192626 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1317/2634] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:21:46.200135 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d273b5-407c-408e-9ec5-8f1c9107d33c None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:46.210176 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d273b5-407c-408e-9ec5-8f1c9107d33c None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:46.215036 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc2b25-7529-4f0c-afcf-1264c877af06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:46.215681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc2b25-7529-4f0c-afcf-1264c877af06 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:46.215990 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc2b25-7529-4f0c-afcf-1264c877af06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:46.216287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc2b25-7529-4f0c-afcf-1264c877af06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:46.216971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc2b25-7529-4f0c-afcf-1264c877af06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:46.218943 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc2b25-7529-4f0c-afcf-1264c877af06 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:46.296198 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1318/2635] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:21:46 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:21:46.300662 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b594e3-e06e-425b-868a-5402372d5042 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:46.311298 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b594e3-e06e-425b-868a-5402372d5042 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:46.371083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ddbfb-60be-4eee-a0fb-2073da5493bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:46.371760 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ddbfb-60be-4eee-a0fb-2073da5493bb admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:46.372049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ddbfb-60be-4eee-a0fb-2073da5493bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:46.372330 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ddbfb-60be-4eee-a0fb-2073da5493bb admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:46.373018 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ddbfb-60be-4eee-a0fb-2073da5493bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:46.375004 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ddbfb-60be-4eee-a0fb-2073da5493bb admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:46.382037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:21:46.453990 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1318/2636] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:21:46 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:21:46.458349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc2b25-7529-4f0c-afcf-1264c877af06 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:46.470044 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc2b25-7529-4f0c-afcf-1264c877af06 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:46.475452 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc9cf5-7765-4994-b7eb-2739829fbc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:46.476376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc9cf5-7765-4994-b7eb-2739829fbc0f admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:46.476660 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc9cf5-7765-4994-b7eb-2739829fbc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:46.476937 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc9cf5-7765-4994-b7eb-2739829fbc0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:46.477802 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc9cf5-7765-4994-b7eb-2739829fbc0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:46.479493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc9cf5-7765-4994-b7eb-2739829fbc0f admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:46.489876 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1319/2637] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:21:46.494173 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ddbfb-60be-4eee-a0fb-2073da5493bb admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:46.504596 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ddbfb-60be-4eee-a0fb-2073da5493bb admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:46.510371 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e36b7-bb2c-4336-8360-f5620f00926b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:46.511077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e36b7-bb2c-4336-8360-f5620f00926b admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:46.511364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e36b7-bb2c-4336-8360-f5620f00926b admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:46.511643 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e36b7-bb2c-4336-8360-f5620f00926b admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:46.512952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e36b7-bb2c-4336-8360-f5620f00926b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:46.515807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e36b7-bb2c-4336-8360-f5620f00926b admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:46.529057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1319/2638] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:21:46.533222 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc9cf5-7765-4994-b7eb-2739829fbc0f admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:46.543873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc9cf5-7765-4994-b7eb-2739829fbc0f admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:46.548601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d7614-a4ea-462f-86fd-9a00600c4792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:46.549470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d7614-a4ea-462f-86fd-9a00600c4792 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:46.549755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d7614-a4ea-462f-86fd-9a00600c4792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:46.550023 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d7614-a4ea-462f-86fd-9a00600c4792 admin admin] PATH_INFO: `/v3/projects/63a23a569a1040e5a8e7b7ebf14e5f2b/users/a54b21796eff432b8a4824142f8bb1b1/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:46.565928 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d7614-a4ea-462f-86fd-9a00600c4792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a23a569a1040e5a8e7b7ebf14e5f2b, user_id=a54b21796eff432b8a4824142f8bb1b1, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:46.568583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d7614-a4ea-462f-86fd-9a00600c4792 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:46.638193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1320/2639] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:21:46 2022] PUT /identity/v3/projects/63a23a569a1040e5a8e7b7ebf14e5f2b/users/a54b21796eff432b8a4824142f8bb1b1/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:21:46.650350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e748d43-2119-45c4-8cd9-b285e48b0c9f None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:46.650766 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e748d43-2119-45c4-8cd9-b285e48b0c9f None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:46.651050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e748d43-2119-45c4-8cd9-b285e48b0c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:46.679262 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-3e748d43-2119-45c4-8cd9-b285e48b0c9f None None] MFA Rules not processed for user `a54b21796eff432b8a4824142f8bb1b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:21:46.681124 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-3e748d43-2119-45c4-8cd9-b285e48b0c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:46.748475 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1320/2640] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:21:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:21:46.753273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d7614-a4ea-462f-86fd-9a00600c4792 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:46.758987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:21:46.822812 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d7614-a4ea-462f-86fd-9a00600c4792 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:46.827542 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b31b45-6409-4262-9536-2ba6f669210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:46.828199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b31b45-6409-4262-9536-2ba6f669210b admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:46.828480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b31b45-6409-4262-9536-2ba6f669210b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:46.828755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b31b45-6409-4262-9536-2ba6f669210b admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:46.829457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b31b45-6409-4262-9536-2ba6f669210b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:46.831438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b31b45-6409-4262-9536-2ba6f669210b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:46.927876 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1321/2641] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:21:46 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:21:46.931929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e748d43-2119-45c4-8cd9-b285e48b0c9f None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:46.943110 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e748d43-2119-45c4-8cd9-b285e48b0c9f None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:47.002769 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d91794-da5f-4288-b609-657c9cf68500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:47.003355 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d91794-da5f-4288-b609-657c9cf68500 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:47.003639 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d91794-da5f-4288-b609-657c9cf68500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:47.003918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d91794-da5f-4288-b609-657c9cf68500 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:47.004615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d91794-da5f-4288-b609-657c9cf68500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:47.007576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d91794-da5f-4288-b609-657c9cf68500 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:47.090066 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1321/2642] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:21:46 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:21:47.094433 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b31b45-6409-4262-9536-2ba6f669210b admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:47.108703 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b31b45-6409-4262-9536-2ba6f669210b admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:47.113575 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d331b8-d32a-496e-b853-9b19eea7c1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:47.114236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d331b8-d32a-496e-b853-9b19eea7c1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:47.114520 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d331b8-d32a-496e-b853-9b19eea7c1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:47.114820 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d331b8-d32a-496e-b853-9b19eea7c1e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:47.115548 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d331b8-d32a-496e-b853-9b19eea7c1e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:47.117375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d331b8-d32a-496e-b853-9b19eea7c1e3 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:47.127500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1322/2643] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:21:47.132669 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d91794-da5f-4288-b609-657c9cf68500 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:47.144078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d91794-da5f-4288-b609-657c9cf68500 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:47.149873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446630f-87c8-49ea-8b45-79b7515e99d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:47.150861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446630f-87c8-49ea-8b45-79b7515e99d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:47.151142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446630f-87c8-49ea-8b45-79b7515e99d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:47.151417 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446630f-87c8-49ea-8b45-79b7515e99d4 admin admin] PATH_INFO: `/v3/projects/94461d04446d479cbecbc6ac3de28b9b/users/a891b656ef86404e9a211b77f5ed0bb3/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:47.167183 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446630f-87c8-49ea-8b45-79b7515e99d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94461d04446d479cbecbc6ac3de28b9b, user_id=a891b656ef86404e9a211b77f5ed0bb3, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:47.174771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446630f-87c8-49ea-8b45-79b7515e99d4 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:47.279288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1322/2644] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:21:47 2022] PUT /identity/v3/projects/94461d04446d479cbecbc6ac3de28b9b/users/a891b656ef86404e9a211b77f5ed0bb3/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:21:47.283438 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d331b8-d32a-496e-b853-9b19eea7c1e3 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:47.294235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d331b8-d32a-496e-b853-9b19eea7c1e3 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:47.351643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca869b40-db91-4404-8934-b3a143ba8f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:47.352358 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca869b40-db91-4404-8934-b3a143ba8f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:47.352652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca869b40-db91-4404-8934-b3a143ba8f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:47.352936 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca869b40-db91-4404-8934-b3a143ba8f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:47.353666 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca869b40-db91-4404-8934-b3a143ba8f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:47.355345 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca869b40-db91-4404-8934-b3a143ba8f31 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:47.365117 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1323/2645] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:21:47.370781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446630f-87c8-49ea-8b45-79b7515e99d4 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:47.381192 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446630f-87c8-49ea-8b45-79b7515e99d4 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:47.387103 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d28cd-4fb5-4c84-8c9d-7618aa6e380f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:47.387978 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d28cd-4fb5-4c84-8c9d-7618aa6e380f admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:47.388262 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d28cd-4fb5-4c84-8c9d-7618aa6e380f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:47.388539 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d28cd-4fb5-4c84-8c9d-7618aa6e380f admin admin] PATH_INFO: `/v3/projects/94461d04446d479cbecbc6ac3de28b9b/users/a891b656ef86404e9a211b77f5ed0bb3/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:47.390512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d28cd-4fb5-4c84-8c9d-7618aa6e380f admin admin] RBAC: Authorizing `identity:create_grant(project_id=94461d04446d479cbecbc6ac3de28b9b, user_id=a891b656ef86404e9a211b77f5ed0bb3, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:47.393207 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d28cd-4fb5-4c84-8c9d-7618aa6e380f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:47.404421 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1323/2646] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:21:47 2022] PUT /identity/v3/projects/94461d04446d479cbecbc6ac3de28b9b/users/a891b656ef86404e9a211b77f5ed0bb3/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:21:47.415333 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212fd46-0300-4ff8-928e-183473cbaf4b None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:47.415726 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212fd46-0300-4ff8-928e-183473cbaf4b None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:47.416020 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212fd46-0300-4ff8-928e-183473cbaf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:47.443318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-c212fd46-0300-4ff8-928e-183473cbaf4b None None] MFA Rules not processed for user `a891b656ef86404e9a211b77f5ed0bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:21:47.445324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c212fd46-0300-4ff8-928e-183473cbaf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:47.510296 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:21:47.513992 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1324/2647] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:21:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:21:47.525673 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d28cd-4fb5-4c84-8c9d-7618aa6e380f admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:47.527468 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-986d28cd-4fb5-4c84-8c9d-7618aa6e380f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:21:47.551427 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:21:47.610168 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d28cd-4fb5-4c84-8c9d-7618aa6e380f admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:47.614388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ee679-5105-413d-97fe-7da0bebe4f4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:47.615064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ee679-5105-413d-97fe-7da0bebe4f4a service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:47.615390 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ee679-5105-413d-97fe-7da0bebe4f4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:47.615627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ee679-5105-413d-97fe-7da0bebe4f4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:47.626204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ee679-5105-413d-97fe-7da0bebe4f4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:47.627699 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ee679-5105-413d-97fe-7da0bebe4f4a service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:47.638705 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1324/2648] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:21:47 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:21:47.887491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212fd46-0300-4ff8-928e-183473cbaf4b None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:21:47.898007 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212fd46-0300-4ff8-928e-183473cbaf4b None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:21:47.901750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835669d-a770-4bf4-9a53-926b7010de72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:21:47.902387 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835669d-a770-4bf4-9a53-926b7010de72 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:21:47.902713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835669d-a770-4bf4-9a53-926b7010de72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:21:47.902987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835669d-a770-4bf4-9a53-926b7010de72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:21:47.960944 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835669d-a770-4bf4-9a53-926b7010de72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:21:47.962388 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835669d-a770-4bf4-9a53-926b7010de72 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:21:47.971445 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1325/2649] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:21:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:21:57.676778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=26 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:21:57.967365 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=28 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:23.531014 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ee679-5105-413d-97fe-7da0bebe4f4a service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:23.532499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-525ee679-5105-413d-97fe-7da0bebe4f4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:23.605313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ee679-5105-413d-97fe-7da0bebe4f4a service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:23.609071 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e0909-1010-4d14-b2b8-ea93c0daa31e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:23.609727 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e0909-1010-4d14-b2b8-ea93c0daa31e service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:23.610012 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e0909-1010-4d14-b2b8-ea93c0daa31e service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:23.610318 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e0909-1010-4d14-b2b8-ea93c0daa31e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:23.612170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-356e0909-1010-4d14-b2b8-ea93c0daa31e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:23.672814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e0909-1010-4d14-b2b8-ea93c0daa31e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:23.674398 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e0909-1010-4d14-b2b8-ea93c0daa31e service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:23.684116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1325/2650] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:22:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:22:30.119611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835669d-a770-4bf4-9a53-926b7010de72 service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:30.121161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e835669d-a770-4bf4-9a53-926b7010de72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:30.140621 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835669d-a770-4bf4-9a53-926b7010de72 service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:30.140621 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f57d1c-8e17-4010-9574-b1c8b44719c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:30.140621 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f57d1c-8e17-4010-9574-b1c8b44719c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:30.140621 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f57d1c-8e17-4010-9574-b1c8b44719c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:30.141191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f57d1c-8e17-4010-9574-b1c8b44719c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:30.142111 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-06f57d1c-8e17-4010-9574-b1c8b44719c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:30.209955 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f57d1c-8e17-4010-9574-b1c8b44719c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:30.211609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f57d1c-8e17-4010-9574-b1c8b44719c3 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:30.222036 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1326/2651] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:22:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:22:30.660186 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c8332-4bad-450b-b819-fb2fbf35875f None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:30.660733 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c8332-4bad-450b-b819-fb2fbf35875f None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:30.660733 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c8332-4bad-450b-b819-fb2fbf35875f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:30.678214 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-111c8332-4bad-450b-b819-fb2fbf35875f None None] MFA Rules not processed for user `5a9a938a9006499a9426af14cb182ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:30.679576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-111c8332-4bad-450b-b819-fb2fbf35875f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:30.740600 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1326/2652] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:31.075136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f57d1c-8e17-4010-9574-b1c8b44719c3 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:31.083966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f57d1c-8e17-4010-9574-b1c8b44719c3 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:31.088271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff38ef7-78a5-4f63-b1b6-5cea83c75c1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:31.088929 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff38ef7-78a5-4f63-b1b6-5cea83c75c1a service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:31.089562 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff38ef7-78a5-4f63-b1b6-5cea83c75c1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:31.089562 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff38ef7-78a5-4f63-b1b6-5cea83c75c1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:31.091320 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0ff38ef7-78a5-4f63-b1b6-5cea83c75c1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:31.101888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff38ef7-78a5-4f63-b1b6-5cea83c75c1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:31.103367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff38ef7-78a5-4f63-b1b6-5cea83c75c1a service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:31.111765 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1327/2653] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:22:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:22:36.867917 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c8332-4bad-450b-b819-fb2fbf35875f None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:36.879747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c8332-4bad-450b-b819-fb2fbf35875f None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:36.885460 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd64d8-2984-41e2-afff-ef19fd64fd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:36.886771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd64d8-2984-41e2-afff-ef19fd64fd5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:36.887322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd64d8-2984-41e2-afff-ef19fd64fd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:36.887783 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd64d8-2984-41e2-afff-ef19fd64fd5d admin admin] PATH_INFO: `/v3/users/a54b21796eff432b8a4824142f8bb1b1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:36.889150 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd64d8-2984-41e2-afff-ef19fd64fd5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a54b21796eff432b8a4824142f8bb1b1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:36.891391 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd64d8-2984-41e2-afff-ef19fd64fd5d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:37.092815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:37.095526 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f7cd64d8-2984-41e2-afff-ef19fd64fd5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a54b21796eff432b8a4824142f8bb1b1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:37.103819 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f7cd64d8-2984-41e2-afff-ef19fd64fd5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a54b21796eff432b8a4824142f8bb1b1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:37.105052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f7cd64d8-2984-41e2-afff-ef19fd64fd5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a54b21796eff432b8a4824142f8bb1b1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:37.118063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f7cd64d8-2984-41e2-afff-ef19fd64fd5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a54b21796eff432b8a4824142f8bb1b1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:37.185702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f7cd64d8-2984-41e2-afff-ef19fd64fd5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a54b21796eff432b8a4824142f8bb1b1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:37.193288 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1327/2654] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:22:36 2022] DELETE /identity/v3/users/a54b21796eff432b8a4824142f8bb1b1 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:37.200529 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acf7a5-3bb7-446f-b9dc-b76b6f80f04f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:37.200529 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acf7a5-3bb7-446f-b9dc-b76b6f80f04f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:37.200529 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acf7a5-3bb7-446f-b9dc-b76b6f80f04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:37.217943 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d4acf7a5-3bb7-446f-b9dc-b76b6f80f04f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:37.219214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d4acf7a5-3bb7-446f-b9dc-b76b6f80f04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:37.287500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1328/2655] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:37.298765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd64d8-2984-41e2-afff-ef19fd64fd5d admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:37.300363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f7cd64d8-2984-41e2-afff-ef19fd64fd5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:37.362826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd64d8-2984-41e2-afff-ef19fd64fd5d admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:37.366438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37adfb7-19c9-4bcf-9a2e-574b66142b82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:37.367888 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37adfb7-19c9-4bcf-9a2e-574b66142b82 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:37.368253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37adfb7-19c9-4bcf-9a2e-574b66142b82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:37.368535 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37adfb7-19c9-4bcf-9a2e-574b66142b82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:37.378036 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37adfb7-19c9-4bcf-9a2e-574b66142b82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:37.379564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37adfb7-19c9-4bcf-9a2e-574b66142b82 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:37.391302 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1328/2656] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:22:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:22:37.828744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acf7a5-3bb7-446f-b9dc-b76b6f80f04f None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:37.830518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d4acf7a5-3bb7-446f-b9dc-b76b6f80f04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:37.852656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acf7a5-3bb7-446f-b9dc-b76b6f80f04f None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:37.868306 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a772819-3b67-4920-a808-07ccccd46dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:37.869539 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a772819-3b67-4920-a808-07ccccd46dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:37.869939 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a772819-3b67-4920-a808-07ccccd46dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:37.870482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a772819-3b67-4920-a808-07ccccd46dff admin admin] PATH_INFO: `/v3/projects/63a23a569a1040e5a8e7b7ebf14e5f2b` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:37.872013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a772819-3b67-4920-a808-07ccccd46dff admin admin] RBAC: Authorizing `identity:delete_project(project_id=63a23a569a1040e5a8e7b7ebf14e5f2b)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:37.874717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a772819-3b67-4920-a808-07ccccd46dff admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:37.995185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0a772819-3b67-4920-a808-07ccccd46dff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63a23a569a1040e5a8e7b7ebf14e5f2b', 'request_id': 'req-0a772819-3b67-4920-a808-07ccccd46dff'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:38.002136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0a772819-3b67-4920-a808-07ccccd46dff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63a23a569a1040e5a8e7b7ebf14e5f2b', 'request_id': 'req-0a772819-3b67-4920-a808-07ccccd46dff'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:38.009200 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0a772819-3b67-4920-a808-07ccccd46dff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63a23a569a1040e5a8e7b7ebf14e5f2b', 'request_id': 'req-0a772819-3b67-4920-a808-07ccccd46dff'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:38.093100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0a772819-3b67-4920-a808-07ccccd46dff admin admin] The token cache is being invalidate because project 63a23a569a1040e5a8e7b7ebf14e5f2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:22:38.093535 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0a772819-3b67-4920-a808-07ccccd46dff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:38.094250 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0a772819-3b67-4920-a808-07ccccd46dff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:38.103904 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1329/2657] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:22:37 2022] DELETE /identity/v3/projects/63a23a569a1040e5a8e7b7ebf14e5f2b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:38.108305 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37adfb7-19c9-4bcf-9a2e-574b66142b82 service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:38.119073 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37adfb7-19c9-4bcf-9a2e-574b66142b82 service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:38.184132 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff988b-43e8-4502-8e63-fa673af01686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:38.185020 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff988b-43e8-4502-8e63-fa673af01686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:38.185315 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff988b-43e8-4502-8e63-fa673af01686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:38.185625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff988b-43e8-4502-8e63-fa673af01686 admin admin] PATH_INFO: `/v3/users/a891b656ef86404e9a211b77f5ed0bb3` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:38.186913 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff988b-43e8-4502-8e63-fa673af01686 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a891b656ef86404e9a211b77f5ed0bb3)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:38.189026 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff988b-43e8-4502-8e63-fa673af01686 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:38.388335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4cff988b-43e8-4502-8e63-fa673af01686 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a891b656ef86404e9a211b77f5ed0bb3'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:38.401451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4cff988b-43e8-4502-8e63-fa673af01686 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a891b656ef86404e9a211b77f5ed0bb3'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:38.402696 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4cff988b-43e8-4502-8e63-fa673af01686 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a891b656ef86404e9a211b77f5ed0bb3'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:38.416833 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:38.424467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4cff988b-43e8-4502-8e63-fa673af01686 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a891b656ef86404e9a211b77f5ed0bb3'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:38.509910 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4cff988b-43e8-4502-8e63-fa673af01686 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a891b656ef86404e9a211b77f5ed0bb3'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:38.526318 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1329/2658] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:22:38 2022] DELETE /identity/v3/users/a891b656ef86404e9a211b77f5ed0bb3 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:39.063154 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a772819-3b67-4920-a808-07ccccd46dff admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:39.064912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0a772819-3b67-4920-a808-07ccccd46dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:39.081907 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:39.088542 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a772819-3b67-4920-a808-07ccccd46dff admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:39.171448 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e065fab-30a0-4a9b-9a5c-33618f43e012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:39.173407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e065fab-30a0-4a9b-9a5c-33618f43e012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:39.173809 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e065fab-30a0-4a9b-9a5c-33618f43e012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:39.174186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e065fab-30a0-4a9b-9a5c-33618f43e012 admin admin] PATH_INFO: `/v3/projects/94461d04446d479cbecbc6ac3de28b9b` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:39.175942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e065fab-30a0-4a9b-9a5c-33618f43e012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94461d04446d479cbecbc6ac3de28b9b)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:39.178132 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e065fab-30a0-4a9b-9a5c-33618f43e012 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:39.331190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-8e065fab-30a0-4a9b-9a5c-33618f43e012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94461d04446d479cbecbc6ac3de28b9b', 'request_id': 'req-8e065fab-30a0-4a9b-9a5c-33618f43e012'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:39.338819 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-8e065fab-30a0-4a9b-9a5c-33618f43e012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94461d04446d479cbecbc6ac3de28b9b', 'request_id': 'req-8e065fab-30a0-4a9b-9a5c-33618f43e012'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:39.346395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-8e065fab-30a0-4a9b-9a5c-33618f43e012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94461d04446d479cbecbc6ac3de28b9b', 'request_id': 'req-8e065fab-30a0-4a9b-9a5c-33618f43e012'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:39.420916 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-8e065fab-30a0-4a9b-9a5c-33618f43e012 admin admin] The token cache is being invalidate because project 94461d04446d479cbecbc6ac3de28b9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:22:39.421244 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-8e065fab-30a0-4a9b-9a5c-33618f43e012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:39.426797 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-8e065fab-30a0-4a9b-9a5c-33618f43e012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:39.426797 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1330/2659] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:22:39 2022] DELETE /identity/v3/projects/94461d04446d479cbecbc6ac3de28b9b => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:39.431083 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0322f-06f5-4ff9-847c-d490183839d6 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:39.431618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0322f-06f5-4ff9-847c-d490183839d6 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:39.431875 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0322f-06f5-4ff9-847c-d490183839d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:39.458861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-1ce0322f-06f5-4ff9-847c-d490183839d6 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:39.458861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1ce0322f-06f5-4ff9-847c-d490183839d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:39.527482 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1330/2660] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:22:39.536484 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b20bc-c9ef-49dd-b4bb-48fd06cf30f1 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:39.536778 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b20bc-c9ef-49dd-b4bb-48fd06cf30f1 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:39.537096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b20bc-c9ef-49dd-b4bb-48fd06cf30f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:39.556471 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-2d3b20bc-c9ef-49dd-b4bb-48fd06cf30f1 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:39.557720 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2d3b20bc-c9ef-49dd-b4bb-48fd06cf30f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:39.565889 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1331/2661] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:39.571161 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0322f-06f5-4ff9-847c-d490183839d6 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:39.581215 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0322f-06f5-4ff9-847c-d490183839d6 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:39.586233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66370e5-0247-43d4-b367-5b655d174d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:39.586947 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66370e5-0247-43d4-b367-5b655d174d5d admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:39.587313 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66370e5-0247-43d4-b367-5b655d174d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:39.587585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66370e5-0247-43d4-b367-5b655d174d5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:39.596529 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66370e5-0247-43d4-b367-5b655d174d5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:22:39.596529 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66370e5-0247-43d4-b367-5b655d174d5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:39.596529 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66370e5-0247-43d4-b367-5b655d174d5d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:39.602935 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1331/2662] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:22:39.607467 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b20bc-c9ef-49dd-b4bb-48fd06cf30f1 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:39.618032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b20bc-c9ef-49dd-b4bb-48fd06cf30f1 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:39.622652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5a796-1d2f-44e2-b4a9-aca49f131091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:39.633604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5a796-1d2f-44e2-b4a9-aca49f131091 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:39.633604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5a796-1d2f-44e2-b4a9-aca49f131091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:39.633604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5a796-1d2f-44e2-b4a9-aca49f131091 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:39.633604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5a796-1d2f-44e2-b4a9-aca49f131091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:39.633604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5a796-1d2f-44e2-b4a9-aca49f131091 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:39.665615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66370e5-0247-43d4-b367-5b655d174d5d admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:39.667459 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d66370e5-0247-43d4-b367-5b655d174d5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:39.681795 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66370e5-0247-43d4-b367-5b655d174d5d admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:39.686845 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12ebc3-c9af-443a-b9ee-bb7326888a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:39.687775 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12ebc3-c9af-443a-b9ee-bb7326888a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:39.688055 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12ebc3-c9af-443a-b9ee-bb7326888a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:39.688332 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12ebc3-c9af-443a-b9ee-bb7326888a64 admin admin] PATH_INFO: `/v3/users/dcfdd71f9b93426bb6b149d2aeb2088e` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:39.689661 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12ebc3-c9af-443a-b9ee-bb7326888a64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcfdd71f9b93426bb6b149d2aeb2088e)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:39.691752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12ebc3-c9af-443a-b9ee-bb7326888a64 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:39.759406 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1332/2663] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:22:39 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:22:39.764256 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5a796-1d2f-44e2-b4a9-aca49f131091 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:39.842382 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5a796-1d2f-44e2-b4a9-aca49f131091 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:39.848227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc9ecc-712e-4ee2-9f02-05c5d20d990a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:39.848833 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc9ecc-712e-4ee2-9f02-05c5d20d990a admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:39.849279 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc9ecc-712e-4ee2-9f02-05c5d20d990a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:39.849565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc9ecc-712e-4ee2-9f02-05c5d20d990a admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:39.850271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc9ecc-712e-4ee2-9f02-05c5d20d990a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:39.852953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc9ecc-712e-4ee2-9f02-05c5d20d990a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:39.941963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-df12ebc3-c9af-443a-b9ee-bb7326888a64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcfdd71f9b93426bb6b149d2aeb2088e'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:39.950041 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-df12ebc3-c9af-443a-b9ee-bb7326888a64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcfdd71f9b93426bb6b149d2aeb2088e'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:39.950827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-df12ebc3-c9af-443a-b9ee-bb7326888a64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcfdd71f9b93426bb6b149d2aeb2088e'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:39.964566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-df12ebc3-c9af-443a-b9ee-bb7326888a64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcfdd71f9b93426bb6b149d2aeb2088e'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:39.968690 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1333/2664] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:22:39 2022] POST /identity/v3/users => generated 286 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:22:39.973060 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc9ecc-712e-4ee2-9f02-05c5d20d990a admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:39.983017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc9ecc-712e-4ee2-9f02-05c5d20d990a admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:39.987962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbc02f-1043-4ca9-a82a-857cd9a92bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:39.988625 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbc02f-1043-4ca9-a82a-857cd9a92bea admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:39.988919 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbc02f-1043-4ca9-a82a-857cd9a92bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:39.989193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbc02f-1043-4ca9-a82a-857cd9a92bea admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:39.989925 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bbc02f-1043-4ca9-a82a-857cd9a92bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:39.991654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bbc02f-1043-4ca9-a82a-857cd9a92bea admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:40.001793 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1334/2665] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:22:40.005854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbc02f-1043-4ca9-a82a-857cd9a92bea admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:40.015902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbc02f-1043-4ca9-a82a-857cd9a92bea admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:40.020672 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74c44e-4640-4c6b-aef2-d34d9e4212e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:40.021311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74c44e-4640-4c6b-aef2-d34d9e4212e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:40.021598 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74c44e-4640-4c6b-aef2-d34d9e4212e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:40.021883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74c44e-4640-4c6b-aef2-d34d9e4212e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:40.022748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74c44e-4640-4c6b-aef2-d34d9e4212e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:40.030109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74c44e-4640-4c6b-aef2-d34d9e4212e2 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:40.035514 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1335/2666] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:22:40.041969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74c44e-4640-4c6b-aef2-d34d9e4212e2 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:40.052998 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74c44e-4640-4c6b-aef2-d34d9e4212e2 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:40.058921 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296bbce-5321-4543-b9a5-35982bf5583a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:40.059890 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296bbce-5321-4543-b9a5-35982bf5583a admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:40.060179 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296bbce-5321-4543-b9a5-35982bf5583a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:40.060445 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296bbce-5321-4543-b9a5-35982bf5583a admin admin] PATH_INFO: `/v3/projects/3bcda5dced1d423c90150d63782c1060/users/eb478bca49854837bf9316f93fe42b24/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:40.076306 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296bbce-5321-4543-b9a5-35982bf5583a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bcda5dced1d423c90150d63782c1060, user_id=eb478bca49854837bf9316f93fe42b24, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:40.080563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-df12ebc3-c9af-443a-b9ee-bb7326888a64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcfdd71f9b93426bb6b149d2aeb2088e'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:40.080692 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296bbce-5321-4543-b9a5-35982bf5583a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:40.083378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1332/2667] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:22:39 2022] DELETE /identity/v3/users/dcfdd71f9b93426bb6b149d2aeb2088e => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:40.212258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1336/2668] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:22:40 2022] PUT /identity/v3/projects/3bcda5dced1d423c90150d63782c1060/users/eb478bca49854837bf9316f93fe42b24/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:22:40.223358 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795a227-e9c5-432c-91f7-157eceb47340 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:40.223807 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795a227-e9c5-432c-91f7-157eceb47340 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:40.224094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795a227-e9c5-432c-91f7-157eceb47340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:40.253304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-7795a227-e9c5-432c-91f7-157eceb47340 None None] MFA Rules not processed for user `eb478bca49854837bf9316f93fe42b24`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:40.254523 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7795a227-e9c5-432c-91f7-157eceb47340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:40.289050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:40.325447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1333/2669] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:22:40.330181 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296bbce-5321-4543-b9a5-35982bf5583a admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:40.331922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2296bbce-5321-4543-b9a5-35982bf5583a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:40.346618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795a227-e9c5-432c-91f7-157eceb47340 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:40.348317 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7795a227-e9c5-432c-91f7-157eceb47340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:40.403816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296bbce-5321-4543-b9a5-35982bf5583a admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:40.408689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fde38-4acb-4463-ab57-05f151ba8ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:40.409366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fde38-4acb-4463-ab57-05f151ba8ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:40.409669 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fde38-4acb-4463-ab57-05f151ba8ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:40.409944 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fde38-4acb-4463-ab57-05f151ba8ee0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:40.410725 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fde38-4acb-4463-ab57-05f151ba8ee0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:40.412648 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fde38-4acb-4463-ab57-05f151ba8ee0 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:40.428979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795a227-e9c5-432c-91f7-157eceb47340 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:40.431699 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:40.434167 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:40.435189 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:40.435589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:40.435738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9 admin admin] PATH_INFO: `/v3/projects/5c88f3beb8bf468f9a30311b203b13cf` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:40.437199 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c88f3beb8bf468f9a30311b203b13cf)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:40.439188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:40.513663 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1337/2670] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:22:40 2022] POST /identity/v3/projects => generated 373 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:22:40.517827 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fde38-4acb-4463-ab57-05f151ba8ee0 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:40.551019 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c88f3beb8bf468f9a30311b203b13cf', 'request_id': 'req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:40.559566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c88f3beb8bf468f9a30311b203b13cf', 'request_id': 'req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:40.591356 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fde38-4acb-4463-ab57-05f151ba8ee0 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:40.651229 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c88f3beb8bf468f9a30311b203b13cf', 'request_id': 'req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:40.654013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7784a33-b73c-47b9-964d-9f91fcfb46c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:40.654742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7784a33-b73c-47b9-964d-9f91fcfb46c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:40.655033 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7784a33-b73c-47b9-964d-9f91fcfb46c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:40.655474 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7784a33-b73c-47b9-964d-9f91fcfb46c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:40.656184 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7784a33-b73c-47b9-964d-9f91fcfb46c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:40.658175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7784a33-b73c-47b9-964d-9f91fcfb46c6 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:40.662958 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9 admin admin] The token cache is being invalidate because project 5c88f3beb8bf468f9a30311b203b13cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:22:40.663747 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:40.664417 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:40.666034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1334/2671] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:22:40 2022] DELETE /identity/v3/projects/5c88f3beb8bf468f9a30311b203b13cf => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:40.670108 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:40.671496 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:40.685524 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ac8bc-7c5a-4fb3-a4ae-d0eaaa561ca9 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:40.690274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b5d61-2a80-43b2-acfd-f1a90a2d74d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:40.691182 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b5d61-2a80-43b2-acfd-f1a90a2d74d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:40.691465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b5d61-2a80-43b2-acfd-f1a90a2d74d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:40.691741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b5d61-2a80-43b2-acfd-f1a90a2d74d3 admin admin] PATH_INFO: `/v3/users/089c15ef31064fee9090685f0302b919` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:40.692939 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b5d61-2a80-43b2-acfd-f1a90a2d74d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=089c15ef31064fee9090685f0302b919)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:40.695204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b5d61-2a80-43b2-acfd-f1a90a2d74d3 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:40.739102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1338/2672] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:22:40 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:22:40.744826 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7784a33-b73c-47b9-964d-9f91fcfb46c6 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:40.746489 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c7784a33-b73c-47b9-964d-9f91fcfb46c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:40.758976 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7784a33-b73c-47b9-964d-9f91fcfb46c6 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:40.763876 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489c5b1-3d53-41dc-afb4-9ad4cfa65b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:40.764541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489c5b1-3d53-41dc-afb4-9ad4cfa65b93 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:40.764831 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489c5b1-3d53-41dc-afb4-9ad4cfa65b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:40.765111 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489c5b1-3d53-41dc-afb4-9ad4cfa65b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:40.765895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489c5b1-3d53-41dc-afb4-9ad4cfa65b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:40.767535 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489c5b1-3d53-41dc-afb4-9ad4cfa65b93 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:40.777748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1339/2673] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:22:40.782138 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489c5b1-3d53-41dc-afb4-9ad4cfa65b93 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:40.792642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489c5b1-3d53-41dc-afb4-9ad4cfa65b93 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:40.797333 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d9da9-3035-46ef-b798-2466f13e600b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:40.798196 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d9da9-3035-46ef-b798-2466f13e600b admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:40.798482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d9da9-3035-46ef-b798-2466f13e600b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:40.798794 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d9da9-3035-46ef-b798-2466f13e600b admin admin] PATH_INFO: `/v3/projects/e931de134857470db1762f493bd845b4/users/6fe3b11059b54e88ae9c3176ff70ecbe/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:40.815037 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d9da9-3035-46ef-b798-2466f13e600b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e931de134857470db1762f493bd845b4, user_id=6fe3b11059b54e88ae9c3176ff70ecbe, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:40.817579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d9da9-3035-46ef-b798-2466f13e600b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:40.906817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-bc3b5d61-2a80-43b2-acfd-f1a90a2d74d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '089c15ef31064fee9090685f0302b919'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:40.914732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-bc3b5d61-2a80-43b2-acfd-f1a90a2d74d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '089c15ef31064fee9090685f0302b919'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:40.915396 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-bc3b5d61-2a80-43b2-acfd-f1a90a2d74d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '089c15ef31064fee9090685f0302b919'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:40.923152 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1340/2674] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:22:40 2022] PUT /identity/v3/projects/e931de134857470db1762f493bd845b4/users/6fe3b11059b54e88ae9c3176ff70ecbe/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:22:40.927175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d9da9-3035-46ef-b798-2466f13e600b admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:40.928293 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-bc3b5d61-2a80-43b2-acfd-f1a90a2d74d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '089c15ef31064fee9090685f0302b919'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:40.962131 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:40.988764 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d9da9-3035-46ef-b798-2466f13e600b admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:40.994424 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a50227-42b6-4885-b900-7bfc91e9b972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:40.995128 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a50227-42b6-4885-b900-7bfc91e9b972 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:40.995416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a50227-42b6-4885-b900-7bfc91e9b972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:40.996654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a50227-42b6-4885-b900-7bfc91e9b972 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:40.996654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a50227-42b6-4885-b900-7bfc91e9b972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:40.998004 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a50227-42b6-4885-b900-7bfc91e9b972 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:41.007961 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1341/2675] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:22:41.012136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a50227-42b6-4885-b900-7bfc91e9b972 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:41.019125 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-bc3b5d61-2a80-43b2-acfd-f1a90a2d74d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '089c15ef31064fee9090685f0302b919'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:41.022241 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a50227-42b6-4885-b900-7bfc91e9b972 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:41.029390 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1335/2676] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:22:40 2022] DELETE /identity/v3/users/089c15ef31064fee9090685f0302b919 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:22:41.029512 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0627ac-9140-4104-a5da-dba0edc0e739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:41.029512 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0627ac-9140-4104-a5da-dba0edc0e739 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:41.029804 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0627ac-9140-4104-a5da-dba0edc0e739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:41.030029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0627ac-9140-4104-a5da-dba0edc0e739 admin admin] PATH_INFO: `/v3/projects/e931de134857470db1762f493bd845b4/users/6fe3b11059b54e88ae9c3176ff70ecbe/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:41.032383 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0627ac-9140-4104-a5da-dba0edc0e739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e931de134857470db1762f493bd845b4, user_id=6fe3b11059b54e88ae9c3176ff70ecbe, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:41.035285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0627ac-9140-4104-a5da-dba0edc0e739 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:41.040395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:41.049102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1342/2677] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:22:41 2022] PUT /identity/v3/projects/e931de134857470db1762f493bd845b4/users/6fe3b11059b54e88ae9c3176ff70ecbe/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:22:41.061013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800da360-1b28-413a-9618-382233b74202 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:41.061455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800da360-1b28-413a-9618-382233b74202 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:41.061760 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800da360-1b28-413a-9618-382233b74202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:41.091945 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-800da360-1b28-413a-9618-382233b74202 None None] MFA Rules not processed for user `6fe3b11059b54e88ae9c3176ff70ecbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:41.093196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-800da360-1b28-413a-9618-382233b74202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:41.165223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1336/2678] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:41.177318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0627ac-9140-4104-a5da-dba0edc0e739 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:41.180043 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0a0627ac-9140-4104-a5da-dba0edc0e739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:41.239026 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0627ac-9140-4104-a5da-dba0edc0e739 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:41.242737 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e309d-51ed-4fcf-bffd-cef67c669dec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:41.243380 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e309d-51ed-4fcf-bffd-cef67c669dec service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:41.243664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e309d-51ed-4fcf-bffd-cef67c669dec service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:41.243939 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e309d-51ed-4fcf-bffd-cef67c669dec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:41.252935 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e309d-51ed-4fcf-bffd-cef67c669dec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:41.254367 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e309d-51ed-4fcf-bffd-cef67c669dec service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:41.264283 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1343/2679] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:22:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:22:41.266581 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800da360-1b28-413a-9618-382233b74202 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:41.271003 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-800da360-1b28-413a-9618-382233b74202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:41.349977 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800da360-1b28-413a-9618-382233b74202 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:41.355538 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1d0a5-66e9-4fe6-abf9-732a5045cf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:41.356416 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1d0a5-66e9-4fe6-abf9-732a5045cf37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:41.356706 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1d0a5-66e9-4fe6-abf9-732a5045cf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:41.356990 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1d0a5-66e9-4fe6-abf9-732a5045cf37 admin admin] PATH_INFO: `/v3/projects/cdde0a089cf840d3b4fbd17947fb872d` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:41.358319 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1d0a5-66e9-4fe6-abf9-732a5045cf37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdde0a089cf840d3b4fbd17947fb872d)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:41.360370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1d0a5-66e9-4fe6-abf9-732a5045cf37 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:41.479364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-28f1d0a5-66e9-4fe6-abf9-732a5045cf37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdde0a089cf840d3b4fbd17947fb872d', 'request_id': 'req-28f1d0a5-66e9-4fe6-abf9-732a5045cf37'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:41.486002 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-28f1d0a5-66e9-4fe6-abf9-732a5045cf37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdde0a089cf840d3b4fbd17947fb872d', 'request_id': 'req-28f1d0a5-66e9-4fe6-abf9-732a5045cf37'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:41.573228 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-28f1d0a5-66e9-4fe6-abf9-732a5045cf37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdde0a089cf840d3b4fbd17947fb872d', 'request_id': 'req-28f1d0a5-66e9-4fe6-abf9-732a5045cf37'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:41.574858 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:41.586169 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-28f1d0a5-66e9-4fe6-abf9-732a5045cf37 admin admin] The token cache is being invalidate because project cdde0a089cf840d3b4fbd17947fb872d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:22:41.586470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-28f1d0a5-66e9-4fe6-abf9-732a5045cf37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:41.587378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-28f1d0a5-66e9-4fe6-abf9-732a5045cf37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:41.589019 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1337/2680] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:22:41 2022] DELETE /identity/v3/projects/cdde0a089cf840d3b4fbd17947fb872d => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:42.361551 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e309d-51ed-4fcf-bffd-cef67c669dec service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:42.363014 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a26e309d-51ed-4fcf-bffd-cef67c669dec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:42.381273 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e309d-51ed-4fcf-bffd-cef67c669dec service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:42.401568 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:42.439190 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea26d8-d1f5-425a-aa97-1a6ade0cb0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:42.440149 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea26d8-d1f5-425a-aa97-1a6ade0cb0d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:42.440437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea26d8-d1f5-425a-aa97-1a6ade0cb0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:42.440713 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea26d8-d1f5-425a-aa97-1a6ade0cb0d7 admin admin] PATH_INFO: `/v3/users/eb478bca49854837bf9316f93fe42b24` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:42.441991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea26d8-d1f5-425a-aa97-1a6ade0cb0d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb478bca49854837bf9316f93fe42b24)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:42.444278 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea26d8-d1f5-425a-aa97-1a6ade0cb0d7 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:42.632788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-eaea26d8-d1f5-425a-aa97-1a6ade0cb0d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb478bca49854837bf9316f93fe42b24'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:42.633433 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-eaea26d8-d1f5-425a-aa97-1a6ade0cb0d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb478bca49854837bf9316f93fe42b24'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:42.634039 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-eaea26d8-d1f5-425a-aa97-1a6ade0cb0d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb478bca49854837bf9316f93fe42b24'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:42.690932 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-eaea26d8-d1f5-425a-aa97-1a6ade0cb0d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb478bca49854837bf9316f93fe42b24'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:42.704562 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-eaea26d8-d1f5-425a-aa97-1a6ade0cb0d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb478bca49854837bf9316f93fe42b24'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:42.718118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1344/2681] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:22:42 2022] DELETE /identity/v3/users/eb478bca49854837bf9316f93fe42b24 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:42.723176 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8cbbb-52c2-4a73-80aa-de497d3d6b92 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:42.723660 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8cbbb-52c2-4a73-80aa-de497d3d6b92 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:42.723780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8cbbb-52c2-4a73-80aa-de497d3d6b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:42.742694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-d4a8cbbb-52c2-4a73-80aa-de497d3d6b92 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:42.744045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d4a8cbbb-52c2-4a73-80aa-de497d3d6b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:42.806237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1338/2682] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:42.817077 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea26d8-d1f5-425a-aa97-1a6ade0cb0d7 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:42.818562 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-eaea26d8-d1f5-425a-aa97-1a6ade0cb0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:42.880331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea26d8-d1f5-425a-aa97-1a6ade0cb0d7 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:42.884336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34021230-2173-4b9f-9e8a-4678c56aca47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:42.884991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34021230-2173-4b9f-9e8a-4678c56aca47 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:42.885275 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34021230-2173-4b9f-9e8a-4678c56aca47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:42.885552 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34021230-2173-4b9f-9e8a-4678c56aca47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:42.895218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34021230-2173-4b9f-9e8a-4678c56aca47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:42.896677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34021230-2173-4b9f-9e8a-4678c56aca47 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:42.908344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1345/2683] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:22:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:22:43.392457 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8cbbb-52c2-4a73-80aa-de497d3d6b92 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:43.393930 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d4a8cbbb-52c2-4a73-80aa-de497d3d6b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:43.414997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8cbbb-52c2-4a73-80aa-de497d3d6b92 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:43.426771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d37914-a48d-47be-b0d6-79efe7a7e2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:43.426771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d37914-a48d-47be-b0d6-79efe7a7e2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:43.426771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d37914-a48d-47be-b0d6-79efe7a7e2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:43.426771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d37914-a48d-47be-b0d6-79efe7a7e2a1 admin admin] PATH_INFO: `/v3/projects/3bcda5dced1d423c90150d63782c1060` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:43.427569 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d37914-a48d-47be-b0d6-79efe7a7e2a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bcda5dced1d423c90150d63782c1060)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:43.427569 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d37914-a48d-47be-b0d6-79efe7a7e2a1 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:43.548162 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-56d37914-a48d-47be-b0d6-79efe7a7e2a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bcda5dced1d423c90150d63782c1060', 'request_id': 'req-56d37914-a48d-47be-b0d6-79efe7a7e2a1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:43.555636 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-56d37914-a48d-47be-b0d6-79efe7a7e2a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bcda5dced1d423c90150d63782c1060', 'request_id': 'req-56d37914-a48d-47be-b0d6-79efe7a7e2a1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:43.624658 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-56d37914-a48d-47be-b0d6-79efe7a7e2a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bcda5dced1d423c90150d63782c1060', 'request_id': 'req-56d37914-a48d-47be-b0d6-79efe7a7e2a1'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:43.638496 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-56d37914-a48d-47be-b0d6-79efe7a7e2a1 admin admin] The token cache is being invalidate because project 3bcda5dced1d423c90150d63782c1060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:22:43.638721 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-56d37914-a48d-47be-b0d6-79efe7a7e2a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:43.639384 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-56d37914-a48d-47be-b0d6-79efe7a7e2a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:43.640948 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1339/2684] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:22:43 2022] DELETE /identity/v3/projects/3bcda5dced1d423c90150d63782c1060 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:43.645161 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34021230-2173-4b9f-9e8a-4678c56aca47 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:43.655501 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34021230-2173-4b9f-9e8a-4678c56aca47 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:43.702917 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:43.717873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d06992-fb36-4ef4-abd4-53165b91ddb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:43.718783 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d06992-fb36-4ef4-abd4-53165b91ddb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:43.719074 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d06992-fb36-4ef4-abd4-53165b91ddb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:43.719354 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d06992-fb36-4ef4-abd4-53165b91ddb4 admin admin] PATH_INFO: `/v3/users/6fe3b11059b54e88ae9c3176ff70ecbe` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:43.720568 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d06992-fb36-4ef4-abd4-53165b91ddb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fe3b11059b54e88ae9c3176ff70ecbe)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:43.722780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d06992-fb36-4ef4-abd4-53165b91ddb4 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:43.931081 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00d06992-fb36-4ef4-abd4-53165b91ddb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fe3b11059b54e88ae9c3176ff70ecbe'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:43.931758 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00d06992-fb36-4ef4-abd4-53165b91ddb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fe3b11059b54e88ae9c3176ff70ecbe'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:43.932375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00d06992-fb36-4ef4-abd4-53165b91ddb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fe3b11059b54e88ae9c3176ff70ecbe'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:44.010813 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00d06992-fb36-4ef4-abd4-53165b91ddb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fe3b11059b54e88ae9c3176ff70ecbe'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:44.024505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-00d06992-fb36-4ef4-abd4-53165b91ddb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fe3b11059b54e88ae9c3176ff70ecbe'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:44.036996 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1346/2685] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:22:43 2022] DELETE /identity/v3/users/6fe3b11059b54e88ae9c3176ff70ecbe => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:45.095135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d37914-a48d-47be-b0d6-79efe7a7e2a1 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:45.096623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-56d37914-a48d-47be-b0d6-79efe7a7e2a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:45.111718 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d37914-a48d-47be-b0d6-79efe7a7e2a1 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:45.175703 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9d1b6-ee69-407b-bcf3-a926a8cdee67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:45.176579 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9d1b6-ee69-407b-bcf3-a926a8cdee67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:45.176863 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9d1b6-ee69-407b-bcf3-a926a8cdee67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:45.177146 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9d1b6-ee69-407b-bcf3-a926a8cdee67 admin admin] PATH_INFO: `/v3/projects/e931de134857470db1762f493bd845b4` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:45.178405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9d1b6-ee69-407b-bcf3-a926a8cdee67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e931de134857470db1762f493bd845b4)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:45.181170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9d1b6-ee69-407b-bcf3-a926a8cdee67 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:45.199381 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:45.306646 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-79f9d1b6-ee69-407b-bcf3-a926a8cdee67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e931de134857470db1762f493bd845b4', 'request_id': 'req-79f9d1b6-ee69-407b-bcf3-a926a8cdee67'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:45.312979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-79f9d1b6-ee69-407b-bcf3-a926a8cdee67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e931de134857470db1762f493bd845b4', 'request_id': 'req-79f9d1b6-ee69-407b-bcf3-a926a8cdee67'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:45.390201 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-79f9d1b6-ee69-407b-bcf3-a926a8cdee67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e931de134857470db1762f493bd845b4', 'request_id': 'req-79f9d1b6-ee69-407b-bcf3-a926a8cdee67'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:45.402478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-79f9d1b6-ee69-407b-bcf3-a926a8cdee67 admin admin] The token cache is being invalidate because project e931de134857470db1762f493bd845b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:22:45.402806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-79f9d1b6-ee69-407b-bcf3-a926a8cdee67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:45.403525 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-79f9d1b6-ee69-407b-bcf3-a926a8cdee67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:45.405391 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1340/2686] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:22:45 2022] DELETE /identity/v3/projects/e931de134857470db1762f493bd845b4 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:45.410563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978b275-06cb-49c9-8c80-0e97c332023f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:45.411128 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978b275-06cb-49c9-8c80-0e97c332023f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:45.411128 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978b275-06cb-49c9-8c80-0e97c332023f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:45.429819 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-4978b275-06cb-49c9-8c80-0e97c332023f None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:45.445456 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-4978b275-06cb-49c9-8c80-0e97c332023f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:45.493910 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1347/2687] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:22:45.503170 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187813e-70e6-4e95-85b8-387297700910 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:45.503554 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187813e-70e6-4e95-85b8-387297700910 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:45.503911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187813e-70e6-4e95-85b8-387297700910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:45.523540 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-4187813e-70e6-4e95-85b8-387297700910 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:45.524901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4187813e-70e6-4e95-85b8-387297700910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:45.533501 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1341/2688] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:45.537851 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978b275-06cb-49c9-8c80-0e97c332023f None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:45.548344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978b275-06cb-49c9-8c80-0e97c332023f None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:45.553428 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5199e-ad37-4132-84c8-60ab164da654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:45.554875 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5199e-ad37-4132-84c8-60ab164da654 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:45.555029 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5199e-ad37-4132-84c8-60ab164da654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:45.555262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5199e-ad37-4132-84c8-60ab164da654 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:45.556014 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5199e-ad37-4132-84c8-60ab164da654 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:22:45.556324 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5199e-ad37-4132-84c8-60ab164da654 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:45.557972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5199e-ad37-4132-84c8-60ab164da654 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:45.571853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1348/2689] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:22:45.576414 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187813e-70e6-4e95-85b8-387297700910 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:45.586325 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187813e-70e6-4e95-85b8-387297700910 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:45.591156 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54e1cf-7ecc-4187-8250-fccd60050685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:45.591834 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54e1cf-7ecc-4187-8250-fccd60050685 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:45.592137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54e1cf-7ecc-4187-8250-fccd60050685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:45.592405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54e1cf-7ecc-4187-8250-fccd60050685 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:45.593109 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54e1cf-7ecc-4187-8250-fccd60050685 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:45.595098 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54e1cf-7ecc-4187-8250-fccd60050685 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:45.694735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1342/2690] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:22:45 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:22:45.701723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5199e-ad37-4132-84c8-60ab164da654 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:45.712364 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5199e-ad37-4132-84c8-60ab164da654 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:45.768069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:45.787991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a785c-147a-43c9-a236-9d85213b5bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:45.787991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a785c-147a-43c9-a236-9d85213b5bbe admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:45.787991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a785c-147a-43c9-a236-9d85213b5bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:45.787991 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a785c-147a-43c9-a236-9d85213b5bbe admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:45.788379 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a785c-147a-43c9-a236-9d85213b5bbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:45.788913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a785c-147a-43c9-a236-9d85213b5bbe admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:45.917099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1349/2691] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:22:45 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:22:45.920479 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54e1cf-7ecc-4187-8250-fccd60050685 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:45.936915 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54e1cf-7ecc-4187-8250-fccd60050685 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:45.942726 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb7d1cd-9c07-4ba2-8032-0443ff0190c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:45.943431 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7d1cd-9c07-4ba2-8032-0443ff0190c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:45.943713 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7d1cd-9c07-4ba2-8032-0443ff0190c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:45.943982 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7d1cd-9c07-4ba2-8032-0443ff0190c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:45.946013 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb7d1cd-9c07-4ba2-8032-0443ff0190c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:45.947811 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb7d1cd-9c07-4ba2-8032-0443ff0190c4 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:45.962496 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1343/2692] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:22:45.967079 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a785c-147a-43c9-a236-9d85213b5bbe admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:45.978685 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a785c-147a-43c9-a236-9d85213b5bbe admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:45.989611 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40edb4c-5810-429f-ade3-a1bcdbefa63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:45.990488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40edb4c-5810-429f-ade3-a1bcdbefa63b admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:45.990920 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40edb4c-5810-429f-ade3-a1bcdbefa63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:45.991311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40edb4c-5810-429f-ade3-a1bcdbefa63b admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:45.992175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40edb4c-5810-429f-ade3-a1bcdbefa63b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:45.994171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40edb4c-5810-429f-ade3-a1bcdbefa63b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:46.008890 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1350/2693] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:22:46.014268 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb7d1cd-9c07-4ba2-8032-0443ff0190c4 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:46.031265 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb7d1cd-9c07-4ba2-8032-0443ff0190c4 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:46.040299 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027122d-f0cf-48f3-9bf0-d2d6d751f8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:46.041241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027122d-f0cf-48f3-9bf0-d2d6d751f8d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:46.041508 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027122d-f0cf-48f3-9bf0-d2d6d751f8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:46.041791 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027122d-f0cf-48f3-9bf0-d2d6d751f8d0 admin admin] PATH_INFO: `/v3/projects/e2edada64c4d479fbaddb405717dc077/users/d05c7c9a50a547eb8cf9f2a28ac9e9f9/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:46.064592 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027122d-f0cf-48f3-9bf0-d2d6d751f8d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2edada64c4d479fbaddb405717dc077, user_id=d05c7c9a50a547eb8cf9f2a28ac9e9f9, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:46.069184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027122d-f0cf-48f3-9bf0-d2d6d751f8d0 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:46.160350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1344/2694] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:22:46 2022] PUT /identity/v3/projects/e2edada64c4d479fbaddb405717dc077/users/d05c7c9a50a547eb8cf9f2a28ac9e9f9/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:46.172006 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87115c-ba4f-4412-8a77-430ac7a20acc None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:46.172297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87115c-ba4f-4412-8a77-430ac7a20acc None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:46.172576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87115c-ba4f-4412-8a77-430ac7a20acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:46.202433 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-ff87115c-ba4f-4412-8a77-430ac7a20acc None None] MFA Rules not processed for user `d05c7c9a50a547eb8cf9f2a28ac9e9f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:46.203603 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ff87115c-ba4f-4412-8a77-430ac7a20acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:46.281688 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1351/2695] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:46.286528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027122d-f0cf-48f3-9bf0-d2d6d751f8d0 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:46.360809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027122d-f0cf-48f3-9bf0-d2d6d751f8d0 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:46.365949 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc896377-cf59-44ce-8278-6d2d0c5e97c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:46.366685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc896377-cf59-44ce-8278-6d2d0c5e97c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:46.366980 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc896377-cf59-44ce-8278-6d2d0c5e97c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:46.367266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc896377-cf59-44ce-8278-6d2d0c5e97c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:46.367974 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc896377-cf59-44ce-8278-6d2d0c5e97c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:46.370252 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc896377-cf59-44ce-8278-6d2d0c5e97c6 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:46.462438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1345/2696] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:22:46 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:22:46.466431 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87115c-ba4f-4412-8a77-430ac7a20acc None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:46.482133 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87115c-ba4f-4412-8a77-430ac7a20acc None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:46.545144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c8463-1607-4301-9287-bd0cc2c6b864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:46.545828 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c8463-1607-4301-9287-bd0cc2c6b864 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:46.546180 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c8463-1607-4301-9287-bd0cc2c6b864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:46.546452 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c8463-1607-4301-9287-bd0cc2c6b864 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:46.547202 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c8463-1607-4301-9287-bd0cc2c6b864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:46.549136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c8463-1607-4301-9287-bd0cc2c6b864 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:46.659558 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1352/2697] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:22:46 2022] POST /identity/v3/users => generated 299 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:22:46.664549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc896377-cf59-44ce-8278-6d2d0c5e97c6 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:46.676179 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc896377-cf59-44ce-8278-6d2d0c5e97c6 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:46.681618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567a386-946c-4c57-8aac-dcad6f1c6dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:46.682441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567a386-946c-4c57-8aac-dcad6f1c6dab admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:46.682841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567a386-946c-4c57-8aac-dcad6f1c6dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:46.683135 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567a386-946c-4c57-8aac-dcad6f1c6dab admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:46.684053 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567a386-946c-4c57-8aac-dcad6f1c6dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:46.685721 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567a386-946c-4c57-8aac-dcad6f1c6dab admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:46.698394 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1346/2698] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:22:46.703747 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c8463-1607-4301-9287-bd0cc2c6b864 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:46.719603 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c8463-1607-4301-9287-bd0cc2c6b864 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:46.724645 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a954f0e-beb0-4879-a8a8-980a6465ab19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:46.725628 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a954f0e-beb0-4879-a8a8-980a6465ab19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:46.725985 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a954f0e-beb0-4879-a8a8-980a6465ab19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:46.726268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a954f0e-beb0-4879-a8a8-980a6465ab19 admin admin] PATH_INFO: `/v3/projects/a5d34436042548ab9e8d6482803ca0ab/users/6ff95df277f64a2e9a3cb875f41fe251/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:46.742836 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a954f0e-beb0-4879-a8a8-980a6465ab19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5d34436042548ab9e8d6482803ca0ab, user_id=6ff95df277f64a2e9a3cb875f41fe251, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:46.745556 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a954f0e-beb0-4879-a8a8-980a6465ab19 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:46.831478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1353/2699] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:22:46 2022] PUT /identity/v3/projects/a5d34436042548ab9e8d6482803ca0ab/users/6ff95df277f64a2e9a3cb875f41fe251/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:22:46.836689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567a386-946c-4c57-8aac-dcad6f1c6dab admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:46.848788 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567a386-946c-4c57-8aac-dcad6f1c6dab admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:46.860227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:46.922675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea148df-3998-470a-bd0d-10e0e01ad735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:46.923733 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea148df-3998-470a-bd0d-10e0e01ad735 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:46.924166 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea148df-3998-470a-bd0d-10e0e01ad735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:46.924595 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea148df-3998-470a-bd0d-10e0e01ad735 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:46.925550 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea148df-3998-470a-bd0d-10e0e01ad735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:46.927403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea148df-3998-470a-bd0d-10e0e01ad735 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:46.938596 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1347/2700] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:22:46.943025 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a954f0e-beb0-4879-a8a8-980a6465ab19 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:46.954838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a954f0e-beb0-4879-a8a8-980a6465ab19 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:46.961110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b99f4-3cc0-4e9f-ba66-ead5ef6c4a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:46.961110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b99f4-3cc0-4e9f-ba66-ead5ef6c4a7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:46.961404 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b99f4-3cc0-4e9f-ba66-ead5ef6c4a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:46.961682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b99f4-3cc0-4e9f-ba66-ead5ef6c4a7a admin admin] PATH_INFO: `/v3/projects/a5d34436042548ab9e8d6482803ca0ab/users/6ff95df277f64a2e9a3cb875f41fe251/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:46.963705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b99f4-3cc0-4e9f-ba66-ead5ef6c4a7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5d34436042548ab9e8d6482803ca0ab, user_id=6ff95df277f64a2e9a3cb875f41fe251, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:46.966414 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b99f4-3cc0-4e9f-ba66-ead5ef6c4a7a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:46.977884 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1354/2701] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:22:46 2022] PUT /identity/v3/projects/a5d34436042548ab9e8d6482803ca0ab/users/6ff95df277f64a2e9a3cb875f41fe251/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:46.989258 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0432f9e-51e1-4ffa-b768-1e5c670b6470 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:46.989560 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0432f9e-51e1-4ffa-b768-1e5c670b6470 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:46.989846 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0432f9e-51e1-4ffa-b768-1e5c670b6470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:47.018062 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-a0432f9e-51e1-4ffa-b768-1e5c670b6470 None None] MFA Rules not processed for user `6ff95df277f64a2e9a3cb875f41fe251`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:47.019373 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a0432f9e-51e1-4ffa-b768-1e5c670b6470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:47.091099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1348/2702] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:47.103639 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b99f4-3cc0-4e9f-ba66-ead5ef6c4a7a admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:47.105393 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a73b99f4-3cc0-4e9f-ba66-ead5ef6c4a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:47.120302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:47.166437 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b99f4-3cc0-4e9f-ba66-ead5ef6c4a7a admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:47.170202 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f57e61-3cb9-4d2c-8103-424be1236cb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:47.170888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f57e61-3cb9-4d2c-8103-424be1236cb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:47.171171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f57e61-3cb9-4d2c-8103-424be1236cb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:47.171470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f57e61-3cb9-4d2c-8103-424be1236cb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:47.182177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f57e61-3cb9-4d2c-8103-424be1236cb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:47.184699 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f57e61-3cb9-4d2c-8103-424be1236cb9 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:47.196422 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1355/2703] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:22:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:22:47.225254 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0432f9e-51e1-4ffa-b768-1e5c670b6470 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:47.289618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0432f9e-51e1-4ffa-b768-1e5c670b6470 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:47.294438 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442db93-c9c8-4f4b-86a5-26d52524da23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:47.295350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442db93-c9c8-4f4b-86a5-26d52524da23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:47.295639 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442db93-c9c8-4f4b-86a5-26d52524da23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:47.295919 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442db93-c9c8-4f4b-86a5-26d52524da23 admin admin] PATH_INFO: `/v3/users/d05c7c9a50a547eb8cf9f2a28ac9e9f9` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:47.297138 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442db93-c9c8-4f4b-86a5-26d52524da23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d05c7c9a50a547eb8cf9f2a28ac9e9f9)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:47.299122 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442db93-c9c8-4f4b-86a5-26d52524da23 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:47.481999 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:47.490609 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7442db93-c9c8-4f4b-86a5-26d52524da23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd05c7c9a50a547eb8cf9f2a28ac9e9f9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:47.497694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7442db93-c9c8-4f4b-86a5-26d52524da23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd05c7c9a50a547eb8cf9f2a28ac9e9f9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:47.498388 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7442db93-c9c8-4f4b-86a5-26d52524da23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd05c7c9a50a547eb8cf9f2a28ac9e9f9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:47.510949 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7442db93-c9c8-4f4b-86a5-26d52524da23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd05c7c9a50a547eb8cf9f2a28ac9e9f9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:47.584626 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7442db93-c9c8-4f4b-86a5-26d52524da23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd05c7c9a50a547eb8cf9f2a28ac9e9f9'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:47.592070 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1349/2704] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:22:47 2022] DELETE /identity/v3/users/d05c7c9a50a547eb8cf9f2a28ac9e9f9 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:47.597117 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f471a-b72e-477d-b611-230aa88418e2 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:47.597632 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f471a-b72e-477d-b611-230aa88418e2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:47.597757 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f471a-b72e-477d-b611-230aa88418e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:47.615782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d78f471a-b72e-477d-b611-230aa88418e2 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:47.617254 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d78f471a-b72e-477d-b611-230aa88418e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:47.679035 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1356/2705] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:47.689165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442db93-c9c8-4f4b-86a5-26d52524da23 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:47.690663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7442db93-c9c8-4f4b-86a5-26d52524da23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:47.750074 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442db93-c9c8-4f4b-86a5-26d52524da23 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:47.756694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5f897-7e62-4973-a332-72b9d4dac48d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:47.757969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5f897-7e62-4973-a332-72b9d4dac48d service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:47.758250 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5f897-7e62-4973-a332-72b9d4dac48d service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:47.758557 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5f897-7e62-4973-a332-72b9d4dac48d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:47.768873 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5f897-7e62-4973-a332-72b9d4dac48d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:47.770339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5f897-7e62-4973-a332-72b9d4dac48d service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:47.780752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1350/2706] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:22:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:22:48.241604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f471a-b72e-477d-b611-230aa88418e2 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:48.244308 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d78f471a-b72e-477d-b611-230aa88418e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:48.259469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f471a-b72e-477d-b611-230aa88418e2 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:48.264698 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ce3ed-e300-4743-abb2-f93d84818233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:48.266973 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ce3ed-e300-4743-abb2-f93d84818233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:48.267113 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ce3ed-e300-4743-abb2-f93d84818233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:48.267462 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ce3ed-e300-4743-abb2-f93d84818233 admin admin] PATH_INFO: `/v3/projects/e2edada64c4d479fbaddb405717dc077` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:48.268561 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ce3ed-e300-4743-abb2-f93d84818233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2edada64c4d479fbaddb405717dc077)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:48.270491 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ce3ed-e300-4743-abb2-f93d84818233 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:48.388768 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-919ce3ed-e300-4743-abb2-f93d84818233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2edada64c4d479fbaddb405717dc077', 'request_id': 'req-919ce3ed-e300-4743-abb2-f93d84818233'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:48.396522 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-919ce3ed-e300-4743-abb2-f93d84818233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2edada64c4d479fbaddb405717dc077', 'request_id': 'req-919ce3ed-e300-4743-abb2-f93d84818233'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:48.403540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-919ce3ed-e300-4743-abb2-f93d84818233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2edada64c4d479fbaddb405717dc077', 'request_id': 'req-919ce3ed-e300-4743-abb2-f93d84818233'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:48.473772 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-919ce3ed-e300-4743-abb2-f93d84818233 admin admin] The token cache is being invalidate because project e2edada64c4d479fbaddb405717dc077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:22:48.474097 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-919ce3ed-e300-4743-abb2-f93d84818233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:48.474749 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-919ce3ed-e300-4743-abb2-f93d84818233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:48.476940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1357/2707] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:22:48 2022] DELETE /identity/v3/projects/e2edada64c4d479fbaddb405717dc077 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:22:48.481290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5f897-7e62-4973-a332-72b9d4dac48d service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:48.491406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5f897-7e62-4973-a332-72b9d4dac48d service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:48.558384 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6e842-c665-4054-a078-f7e5d5428897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:48.559296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6e842-c665-4054-a078-f7e5d5428897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:48.559649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6e842-c665-4054-a078-f7e5d5428897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:48.559956 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6e842-c665-4054-a078-f7e5d5428897 admin admin] PATH_INFO: `/v3/users/6ff95df277f64a2e9a3cb875f41fe251` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:48.561171 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b6e842-c665-4054-a078-f7e5d5428897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff95df277f64a2e9a3cb875f41fe251)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:48.564252 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b6e842-c665-4054-a078-f7e5d5428897 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:48.762840 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-d6b6e842-c665-4054-a078-f7e5d5428897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff95df277f64a2e9a3cb875f41fe251'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:48.770518 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-d6b6e842-c665-4054-a078-f7e5d5428897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff95df277f64a2e9a3cb875f41fe251'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:48.771333 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-d6b6e842-c665-4054-a078-f7e5d5428897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff95df277f64a2e9a3cb875f41fe251'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:48.774109 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:48.785295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-d6b6e842-c665-4054-a078-f7e5d5428897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff95df277f64a2e9a3cb875f41fe251'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:48.858624 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-d6b6e842-c665-4054-a078-f7e5d5428897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff95df277f64a2e9a3cb875f41fe251'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:48.865861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1351/2708] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:22:48 2022] DELETE /identity/v3/users/6ff95df277f64a2e9a3cb875f41fe251 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:49.328591 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ce3ed-e300-4743-abb2-f93d84818233 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:49.330297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-919ce3ed-e300-4743-abb2-f93d84818233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:49.347445 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ce3ed-e300-4743-abb2-f93d84818233 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:49.397022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:49.432556 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03a779-727a-45fa-b519-08c2a4204a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:49.433430 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03a779-727a-45fa-b519-08c2a4204a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:49.433717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03a779-727a-45fa-b519-08c2a4204a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:49.434004 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03a779-727a-45fa-b519-08c2a4204a54 admin admin] PATH_INFO: `/v3/projects/a5d34436042548ab9e8d6482803ca0ab` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:49.435233 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03a779-727a-45fa-b519-08c2a4204a54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5d34436042548ab9e8d6482803ca0ab)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:49.437145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03a779-727a-45fa-b519-08c2a4204a54 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:49.572949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-fa03a779-727a-45fa-b519-08c2a4204a54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5d34436042548ab9e8d6482803ca0ab', 'request_id': 'req-fa03a779-727a-45fa-b519-08c2a4204a54'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:49.579136 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-fa03a779-727a-45fa-b519-08c2a4204a54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5d34436042548ab9e8d6482803ca0ab', 'request_id': 'req-fa03a779-727a-45fa-b519-08c2a4204a54'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:49.584609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-fa03a779-727a-45fa-b519-08c2a4204a54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5d34436042548ab9e8d6482803ca0ab', 'request_id': 'req-fa03a779-727a-45fa-b519-08c2a4204a54'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:49.656172 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-fa03a779-727a-45fa-b519-08c2a4204a54 admin admin] The token cache is being invalidate because project a5d34436042548ab9e8d6482803ca0ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:22:49.656488 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-fa03a779-727a-45fa-b519-08c2a4204a54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:49.657157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-fa03a779-727a-45fa-b519-08c2a4204a54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:49.658891 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1358/2709] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:22:49 2022] DELETE /identity/v3/projects/a5d34436042548ab9e8d6482803ca0ab => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:49.664707 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b6da3-f66d-4208-8c13-7277aa5b1aba None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:49.665231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b6da3-f66d-4208-8c13-7277aa5b1aba None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:49.665364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b6da3-f66d-4208-8c13-7277aa5b1aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:49.684658 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-6c8b6da3-f66d-4208-8c13-7277aa5b1aba None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:49.685889 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-6c8b6da3-f66d-4208-8c13-7277aa5b1aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:49.760613 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1352/2710] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:49.768860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad635cf-7392-454c-b385-193ac12bda7e None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:49.769145 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad635cf-7392-454c-b385-193ac12bda7e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:49.769425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad635cf-7392-454c-b385-193ac12bda7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:49.789177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-4ad635cf-7392-454c-b385-193ac12bda7e None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:49.790762 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-4ad635cf-7392-454c-b385-193ac12bda7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:49.798867 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1359/2711] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:49.803505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b6da3-f66d-4208-8c13-7277aa5b1aba None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:49.816665 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b6da3-f66d-4208-8c13-7277aa5b1aba None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:49.822073 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed31c35-78b9-437a-8774-3665bab4043f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:49.822782 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed31c35-78b9-437a-8774-3665bab4043f admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:49.823126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed31c35-78b9-437a-8774-3665bab4043f admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:49.823355 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed31c35-78b9-437a-8774-3665bab4043f admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:49.824090 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed31c35-78b9-437a-8774-3665bab4043f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:22:49.824399 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed31c35-78b9-437a-8774-3665bab4043f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:49.826034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed31c35-78b9-437a-8774-3665bab4043f admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:49.838892 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1353/2712] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:22:49.843858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad635cf-7392-454c-b385-193ac12bda7e None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:49.855358 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad635cf-7392-454c-b385-193ac12bda7e None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:49.863031 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352e9d7-1a73-4229-ba2e-50a1011ecb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:49.863694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352e9d7-1a73-4229-ba2e-50a1011ecb62 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:49.863974 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352e9d7-1a73-4229-ba2e-50a1011ecb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:49.864252 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352e9d7-1a73-4229-ba2e-50a1011ecb62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:49.864953 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f352e9d7-1a73-4229-ba2e-50a1011ecb62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:49.867786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f352e9d7-1a73-4229-ba2e-50a1011ecb62 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:49.989988 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1360/2713] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:22:49 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:22:49.997276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed31c35-78b9-437a-8774-3665bab4043f admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:50.007584 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed31c35-78b9-437a-8774-3665bab4043f admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:50.065820 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a39a2-f559-4ae7-9cd4-d3138927d363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:50.066503 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a39a2-f559-4ae7-9cd4-d3138927d363 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:50.066829 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a39a2-f559-4ae7-9cd4-d3138927d363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:50.067111 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a39a2-f559-4ae7-9cd4-d3138927d363 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:50.067824 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a39a2-f559-4ae7-9cd4-d3138927d363 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:50.069915 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a39a2-f559-4ae7-9cd4-d3138927d363 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:50.333527 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1354/2714] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:22:49 2022] POST /identity/v3/users => generated 290 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:22:50.340905 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352e9d7-1a73-4229-ba2e-50a1011ecb62 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:50.352787 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352e9d7-1a73-4229-ba2e-50a1011ecb62 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:50.357370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afad9d-3f61-443f-812a-ae92f2a12ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:50.358072 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afad9d-3f61-443f-812a-ae92f2a12ace admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:50.358358 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afad9d-3f61-443f-812a-ae92f2a12ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:50.358668 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afad9d-3f61-443f-812a-ae92f2a12ace admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:50.359394 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afad9d-3f61-443f-812a-ae92f2a12ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:50.361014 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afad9d-3f61-443f-812a-ae92f2a12ace admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:50.372709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1361/2715] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:22:50.377019 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a39a2-f559-4ae7-9cd4-d3138927d363 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:50.387838 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a39a2-f559-4ae7-9cd4-d3138927d363 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:50.392520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc5cf4-6408-4ee4-922f-c2163656e98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:50.394975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc5cf4-6408-4ee4-922f-c2163656e98a admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:50.394975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc5cf4-6408-4ee4-922f-c2163656e98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:50.394975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc5cf4-6408-4ee4-922f-c2163656e98a admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:50.394975 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc5cf4-6408-4ee4-922f-c2163656e98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:50.396086 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc5cf4-6408-4ee4-922f-c2163656e98a admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:50.409060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1355/2716] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:22:50.414261 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afad9d-3f61-443f-812a-ae92f2a12ace admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:50.424753 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afad9d-3f61-443f-812a-ae92f2a12ace admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:50.429533 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77b8bc-3d09-454e-a272-fa969f349b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:50.430421 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77b8bc-3d09-454e-a272-fa969f349b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:50.430732 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77b8bc-3d09-454e-a272-fa969f349b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:50.431018 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77b8bc-3d09-454e-a272-fa969f349b32 admin admin] PATH_INFO: `/v3/projects/e5c1fae627f44517ae6b8a5d89fee935/users/a671f9519c1b4bc3b3ea95d117579559/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:50.446774 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77b8bc-3d09-454e-a272-fa969f349b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5c1fae627f44517ae6b8a5d89fee935, user_id=a671f9519c1b4bc3b3ea95d117579559, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:50.450458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77b8bc-3d09-454e-a272-fa969f349b32 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:50.532479 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1362/2717] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:22:50 2022] PUT /identity/v3/projects/e5c1fae627f44517ae6b8a5d89fee935/users/a671f9519c1b4bc3b3ea95d117579559/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:50.546676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b1bec-cd6e-4b6c-a5e2-88b03827e0eb None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:50.546979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b1bec-cd6e-4b6c-a5e2-88b03827e0eb None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:50.547267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b1bec-cd6e-4b6c-a5e2-88b03827e0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:50.575546 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-dd7b1bec-cd6e-4b6c-a5e2-88b03827e0eb None None] MFA Rules not processed for user `a671f9519c1b4bc3b3ea95d117579559`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:50.576853 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-dd7b1bec-cd6e-4b6c-a5e2-88b03827e0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:50.607155 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:50.643659 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1356/2718] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:50.649185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77b8bc-3d09-454e-a272-fa969f349b32 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:50.712680 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77b8bc-3d09-454e-a272-fa969f349b32 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:50.717523 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793de67-c984-4be6-812f-d044c9d42170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:50.718191 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793de67-c984-4be6-812f-d044c9d42170 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:50.718480 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793de67-c984-4be6-812f-d044c9d42170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:50.718795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793de67-c984-4be6-812f-d044c9d42170 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:50.719507 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793de67-c984-4be6-812f-d044c9d42170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:50.721419 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793de67-c984-4be6-812f-d044c9d42170 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:50.730033 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:50.809736 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1363/2719] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:22:50 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:22:50.814205 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b1bec-cd6e-4b6c-a5e2-88b03827e0eb None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:50.825947 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b1bec-cd6e-4b6c-a5e2-88b03827e0eb None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:50.885605 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b4395-6870-47cd-9a87-36c124973b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:50.886210 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b4395-6870-47cd-9a87-36c124973b02 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:50.886490 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b4395-6870-47cd-9a87-36c124973b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:50.886801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b4395-6870-47cd-9a87-36c124973b02 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:50.887527 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b4395-6870-47cd-9a87-36c124973b02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:50.889511 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b4395-6870-47cd-9a87-36c124973b02 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:50.997125 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1357/2720] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:22:50 2022] POST /identity/v3/users => generated 296 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:22:51.001384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793de67-c984-4be6-812f-d044c9d42170 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:51.013433 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793de67-c984-4be6-812f-d044c9d42170 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:51.018218 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e791510-cbc6-4fe0-8bc2-71a5075617e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:51.018870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e791510-cbc6-4fe0-8bc2-71a5075617e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:51.019155 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e791510-cbc6-4fe0-8bc2-71a5075617e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:51.019434 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e791510-cbc6-4fe0-8bc2-71a5075617e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:51.020159 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e791510-cbc6-4fe0-8bc2-71a5075617e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:51.021801 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e791510-cbc6-4fe0-8bc2-71a5075617e6 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:51.033077 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1364/2721] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:22:51.037400 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b4395-6870-47cd-9a87-36c124973b02 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:51.047637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b4395-6870-47cd-9a87-36c124973b02 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:51.052559 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e9612-e9ae-4fc2-86ec-53b5cbb78e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:51.053450 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e9612-e9ae-4fc2-86ec-53b5cbb78e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:51.053736 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e9612-e9ae-4fc2-86ec-53b5cbb78e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:51.054507 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e9612-e9ae-4fc2-86ec-53b5cbb78e45 admin admin] PATH_INFO: `/v3/projects/2765f5ff44784088b4652905fa2a5931/users/59964211b44842f1a0b44927ab690d44/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:51.070262 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e9612-e9ae-4fc2-86ec-53b5cbb78e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2765f5ff44784088b4652905fa2a5931, user_id=59964211b44842f1a0b44927ab690d44, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:51.073034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e9612-e9ae-4fc2-86ec-53b5cbb78e45 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:51.162043 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1358/2722] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:22:51 2022] PUT /identity/v3/projects/2765f5ff44784088b4652905fa2a5931/users/59964211b44842f1a0b44927ab690d44/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:51.166805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e791510-cbc6-4fe0-8bc2-71a5075617e6 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:51.176964 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e791510-cbc6-4fe0-8bc2-71a5075617e6 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:51.234083 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91ac8c-afcc-4c56-b650-21ba3c2d2a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:51.234788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91ac8c-afcc-4c56-b650-21ba3c2d2a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:51.235075 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91ac8c-afcc-4c56-b650-21ba3c2d2a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:51.235357 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91ac8c-afcc-4c56-b650-21ba3c2d2a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:51.236080 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91ac8c-afcc-4c56-b650-21ba3c2d2a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:51.237689 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91ac8c-afcc-4c56-b650-21ba3c2d2a20 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:51.248084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1365/2723] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:22:51.252544 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e9612-e9ae-4fc2-86ec-53b5cbb78e45 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:51.263041 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e9612-e9ae-4fc2-86ec-53b5cbb78e45 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:51.267694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4844273-49b6-4150-8d80-240a8d241662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:51.268580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4844273-49b6-4150-8d80-240a8d241662 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:51.268867 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4844273-49b6-4150-8d80-240a8d241662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:51.269145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4844273-49b6-4150-8d80-240a8d241662 admin admin] PATH_INFO: `/v3/projects/2765f5ff44784088b4652905fa2a5931/users/59964211b44842f1a0b44927ab690d44/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:51.271593 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4844273-49b6-4150-8d80-240a8d241662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2765f5ff44784088b4652905fa2a5931, user_id=59964211b44842f1a0b44927ab690d44, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:51.274916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4844273-49b6-4150-8d80-240a8d241662 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:51.286267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1359/2724] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:22:51 2022] PUT /identity/v3/projects/2765f5ff44784088b4652905fa2a5931/users/59964211b44842f1a0b44927ab690d44/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:22:51.299268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff607f-25dd-422c-b064-51346b7cd09d None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:51.299613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff607f-25dd-422c-b064-51346b7cd09d None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:51.307721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff607f-25dd-422c-b064-51346b7cd09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:51.327042 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-26ff607f-25dd-422c-b064-51346b7cd09d None None] MFA Rules not processed for user `59964211b44842f1a0b44927ab690d44`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:51.328270 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-26ff607f-25dd-422c-b064-51346b7cd09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:51.395233 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1366/2725] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:51.408183 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4844273-49b6-4150-8d80-240a8d241662 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:51.409867 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-c4844273-49b6-4150-8d80-240a8d241662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:51.467979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4844273-49b6-4150-8d80-240a8d241662 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:51.471594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06da10-8dcd-42f9-b9ea-16877fc680d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:51.472502 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06da10-8dcd-42f9-b9ea-16877fc680d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:51.472789 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06da10-8dcd-42f9-b9ea-16877fc680d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:51.473067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06da10-8dcd-42f9-b9ea-16877fc680d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:51.483079 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06da10-8dcd-42f9-b9ea-16877fc680d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:51.484574 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06da10-8dcd-42f9-b9ea-16877fc680d5 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:51.494425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1360/2726] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:22:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:22:52.385116 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff607f-25dd-422c-b064-51346b7cd09d None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:52.394571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff607f-25dd-422c-b064-51346b7cd09d None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:52.400064 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958eb7d-6638-4c9f-b404-ce104fca2484 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59964211b44842f1a0b44927ab690d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2765f5ff44784088b4652905fa2a5931', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:52.400717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958eb7d-6638-4c9f-b404-ce104fca2484 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:52.401013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958eb7d-6638-4c9f-b404-ce104fca2484 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:52.401295 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958eb7d-6638-4c9f-b404-ce104fca2484 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:52.402032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958eb7d-6638-4c9f-b404-ce104fca2484 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:22:52.402349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958eb7d-6638-4c9f-b404-ce104fca2484 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:52.404034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958eb7d-6638-4c9f-b404-ce104fca2484 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:52.416621 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1367/2727] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:22:52.420771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06da10-8dcd-42f9-b9ea-16877fc680d5 service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:52.430615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06da10-8dcd-42f9-b9ea-16877fc680d5 service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:52.435208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a3783-0572-4faf-83da-388671a559ed tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59964211b44842f1a0b44927ab690d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2765f5ff44784088b4652905fa2a5931', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:52.435870 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a3783-0572-4faf-83da-388671a559ed tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:52.436175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a3783-0572-4faf-83da-388671a559ed tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:52.436456 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a3783-0572-4faf-83da-388671a559ed tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:52.437184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a3783-0572-4faf-83da-388671a559ed tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:52.439137 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a3783-0572-4faf-83da-388671a559ed tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:52.524026 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1361/2728] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:22:52 2022] POST /identity/v3/projects => generated 354 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:22:52.528550 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958eb7d-6638-4c9f-b404-ce104fca2484 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:52.539481 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958eb7d-6638-4c9f-b404-ce104fca2484 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:52.583266 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:52.596731 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c6a87-fc94-4f45-9350-72ba59a205e4 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59964211b44842f1a0b44927ab690d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2765f5ff44784088b4652905fa2a5931', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:52.597588 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c6a87-fc94-4f45-9350-72ba59a205e4 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:52.597885 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c6a87-fc94-4f45-9350-72ba59a205e4 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:52.598169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c6a87-fc94-4f45-9350-72ba59a205e4 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:52.598942 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c6a87-fc94-4f45-9350-72ba59a205e4 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:22:52.599253 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c6a87-fc94-4f45-9350-72ba59a205e4 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:52.601534 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c6a87-fc94-4f45-9350-72ba59a205e4 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:52.614084 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1368/2729] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:22:52.651096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a3783-0572-4faf-83da-388671a559ed tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:52.657799 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:52.661556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a3783-0572-4faf-83da-388671a559ed tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:52.666892 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c857a-9519-4e8c-8dae-9ed59015e84c tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59964211b44842f1a0b44927ab690d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2765f5ff44784088b4652905fa2a5931', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:52.674057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c857a-9519-4e8c-8dae-9ed59015e84c tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:52.674057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c857a-9519-4e8c-8dae-9ed59015e84c tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:52.674057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c857a-9519-4e8c-8dae-9ed59015e84c tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] PATH_INFO: `/v3/projects/24e42cc3ce2349e58f6c50c7a7b30ad4` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:52.674057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54c857a-9519-4e8c-8dae-9ed59015e84c tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] RBAC: Authorizing `identity:delete_project(project_id=24e42cc3ce2349e58f6c50c7a7b30ad4)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:52.674665 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54c857a-9519-4e8c-8dae-9ed59015e84c tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:52.802096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c54c857a-9519-4e8c-8dae-9ed59015e84c tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24e42cc3ce2349e58f6c50c7a7b30ad4', 'request_id': 'req-c54c857a-9519-4e8c-8dae-9ed59015e84c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:52.808637 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c54c857a-9519-4e8c-8dae-9ed59015e84c tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24e42cc3ce2349e58f6c50c7a7b30ad4', 'request_id': 'req-c54c857a-9519-4e8c-8dae-9ed59015e84c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:52.881616 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c54c857a-9519-4e8c-8dae-9ed59015e84c tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24e42cc3ce2349e58f6c50c7a7b30ad4', 'request_id': 'req-c54c857a-9519-4e8c-8dae-9ed59015e84c'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:52.894836 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c54c857a-9519-4e8c-8dae-9ed59015e84c tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] The token cache is being invalidate because project 24e42cc3ce2349e58f6c50c7a7b30ad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:22:52.895123 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c54c857a-9519-4e8c-8dae-9ed59015e84c tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:52.895796 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-c54c857a-9519-4e8c-8dae-9ed59015e84c tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:52.897381 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1362/2730] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:22:52 2022] DELETE /identity/v3/projects/24e42cc3ce2349e58f6c50c7a7b30ad4 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:53.532314 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c6a87-fc94-4f45-9350-72ba59a205e4 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:53.534204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7f5c6a87-fc94-4f45-9350-72ba59a205e4 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:53.598792 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c6a87-fc94-4f45-9350-72ba59a205e4 tempest-VolumeQuotaClassesTest-132047687 tempest-VolumeQuotaClassesTest-132047687-project-admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:53.656052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91ad03-c467-4f36-8936-4e6d78f4a023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:53.656826 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91ad03-c467-4f36-8936-4e6d78f4a023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:53.657109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91ad03-c467-4f36-8936-4e6d78f4a023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:53.657385 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91ad03-c467-4f36-8936-4e6d78f4a023 admin admin] PATH_INFO: `/v3/users/a671f9519c1b4bc3b3ea95d117579559` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:53.658637 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91ad03-c467-4f36-8936-4e6d78f4a023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a671f9519c1b4bc3b3ea95d117579559)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:53.660580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91ad03-c467-4f36-8936-4e6d78f4a023 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:53.886511 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ce91ad03-c467-4f36-8936-4e6d78f4a023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a671f9519c1b4bc3b3ea95d117579559'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:53.887235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ce91ad03-c467-4f36-8936-4e6d78f4a023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a671f9519c1b4bc3b3ea95d117579559'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:53.887844 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ce91ad03-c467-4f36-8936-4e6d78f4a023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a671f9519c1b4bc3b3ea95d117579559'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:53.982414 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ce91ad03-c467-4f36-8936-4e6d78f4a023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a671f9519c1b4bc3b3ea95d117579559'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:53.995660 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-ce91ad03-c467-4f36-8936-4e6d78f4a023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a671f9519c1b4bc3b3ea95d117579559'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:54.008193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1369/2731] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:22:53 2022] DELETE /identity/v3/users/a671f9519c1b4bc3b3ea95d117579559 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:54.012880 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211585e-9836-4d32-9134-9c660d3d0fda None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:54.013360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211585e-9836-4d32-9134-9c660d3d0fda None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:54.013477 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211585e-9836-4d32-9134-9c660d3d0fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:54.031750 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-e211585e-9836-4d32-9134-9c660d3d0fda None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:54.032981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e211585e-9836-4d32-9134-9c660d3d0fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:54.095577 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1363/2732] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:54.106881 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91ad03-c467-4f36-8936-4e6d78f4a023 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:54.108319 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ce91ad03-c467-4f36-8936-4e6d78f4a023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:54.118458 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:54.168170 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91ad03-c467-4f36-8936-4e6d78f4a023 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:54.171461 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e2200-cea3-4fa8-b586-d6f96424cdda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:54.176976 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e2200-cea3-4fa8-b586-d6f96424cdda service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:54.176976 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e2200-cea3-4fa8-b586-d6f96424cdda service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:54.176976 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e2200-cea3-4fa8-b586-d6f96424cdda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:54.183150 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e2200-cea3-4fa8-b586-d6f96424cdda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:54.184529 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e2200-cea3-4fa8-b586-d6f96424cdda service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:54.194200 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1370/2733] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:22:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:22:54.647317 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211585e-9836-4d32-9134-9c660d3d0fda None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:54.648944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e211585e-9836-4d32-9134-9c660d3d0fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:54.661248 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211585e-9836-4d32-9134-9c660d3d0fda None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:54.666311 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d01474-275f-4ce8-bc7b-b54ed842e1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:54.667231 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d01474-275f-4ce8-bc7b-b54ed842e1c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:54.667516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d01474-275f-4ce8-bc7b-b54ed842e1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:54.667793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d01474-275f-4ce8-bc7b-b54ed842e1c5 admin admin] PATH_INFO: `/v3/projects/e5c1fae627f44517ae6b8a5d89fee935` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:54.669005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d01474-275f-4ce8-bc7b-b54ed842e1c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5c1fae627f44517ae6b8a5d89fee935)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:54.670990 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d01474-275f-4ce8-bc7b-b54ed842e1c5 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:54.785370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-33d01474-275f-4ce8-bc7b-b54ed842e1c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5c1fae627f44517ae6b8a5d89fee935', 'request_id': 'req-33d01474-275f-4ce8-bc7b-b54ed842e1c5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:54.791817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-33d01474-275f-4ce8-bc7b-b54ed842e1c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5c1fae627f44517ae6b8a5d89fee935', 'request_id': 'req-33d01474-275f-4ce8-bc7b-b54ed842e1c5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:54.868689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-33d01474-275f-4ce8-bc7b-b54ed842e1c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5c1fae627f44517ae6b8a5d89fee935', 'request_id': 'req-33d01474-275f-4ce8-bc7b-b54ed842e1c5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:54.880052 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-33d01474-275f-4ce8-bc7b-b54ed842e1c5 admin admin] The token cache is being invalidate because project e5c1fae627f44517ae6b8a5d89fee935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:22:54.880465 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-33d01474-275f-4ce8-bc7b-b54ed842e1c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:54.880983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-33d01474-275f-4ce8-bc7b-b54ed842e1c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:54.883370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1364/2734] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:22:54 2022] DELETE /identity/v3/projects/e5c1fae627f44517ae6b8a5d89fee935 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:54.887869 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e2200-cea3-4fa8-b586-d6f96424cdda service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:54.897798 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e2200-cea3-4fa8-b586-d6f96424cdda service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:54.958925 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fa431-25d8-4b09-af05-f89e06d1e9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:54.959806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fa431-25d8-4b09-af05-f89e06d1e9d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:54.960126 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fa431-25d8-4b09-af05-f89e06d1e9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:54.960375 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fa431-25d8-4b09-af05-f89e06d1e9d8 admin admin] PATH_INFO: `/v3/users/59964211b44842f1a0b44927ab690d44` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:54.961594 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fa431-25d8-4b09-af05-f89e06d1e9d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59964211b44842f1a0b44927ab690d44)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:54.963571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fa431-25d8-4b09-af05-f89e06d1e9d8 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:55.159616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5c5fa431-25d8-4b09-af05-f89e06d1e9d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59964211b44842f1a0b44927ab690d44'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:55.160183 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5c5fa431-25d8-4b09-af05-f89e06d1e9d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59964211b44842f1a0b44927ab690d44'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:55.160806 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5c5fa431-25d8-4b09-af05-f89e06d1e9d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59964211b44842f1a0b44927ab690d44'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:55.229788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5c5fa431-25d8-4b09-af05-f89e06d1e9d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59964211b44842f1a0b44927ab690d44'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:55.242679 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-5c5fa431-25d8-4b09-af05-f89e06d1e9d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59964211b44842f1a0b44927ab690d44'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:55.255374 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1371/2735] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:22:54 2022] DELETE /identity/v3/users/59964211b44842f1a0b44927ab690d44 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:22:55.700605 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d01474-275f-4ce8-bc7b-b54ed842e1c5 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:55.702165 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-33d01474-275f-4ce8-bc7b-b54ed842e1c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:55.718227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d01474-275f-4ce8-bc7b-b54ed842e1c5 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:55.729882 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:55.780162 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0c356-2aa9-4734-b114-1e24cdf822d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:55.781035 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0c356-2aa9-4734-b114-1e24cdf822d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:55.781323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0c356-2aa9-4734-b114-1e24cdf822d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:55.781602 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0c356-2aa9-4734-b114-1e24cdf822d8 admin admin] PATH_INFO: `/v3/projects/2765f5ff44784088b4652905fa2a5931` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:55.782895 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0c356-2aa9-4734-b114-1e24cdf822d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2765f5ff44784088b4652905fa2a5931)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:55.784994 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0c356-2aa9-4734-b114-1e24cdf822d8 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:55.940757 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4ee0c356-2aa9-4734-b114-1e24cdf822d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2765f5ff44784088b4652905fa2a5931', 'request_id': 'req-4ee0c356-2aa9-4734-b114-1e24cdf822d8'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:55.955006 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4ee0c356-2aa9-4734-b114-1e24cdf822d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2765f5ff44784088b4652905fa2a5931', 'request_id': 'req-4ee0c356-2aa9-4734-b114-1e24cdf822d8'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:56.030632 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4ee0c356-2aa9-4734-b114-1e24cdf822d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2765f5ff44784088b4652905fa2a5931', 'request_id': 'req-4ee0c356-2aa9-4734-b114-1e24cdf822d8'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:56.043378 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4ee0c356-2aa9-4734-b114-1e24cdf822d8 admin admin] The token cache is being invalidate because project 2765f5ff44784088b4652905fa2a5931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:22:56.043676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4ee0c356-2aa9-4734-b114-1e24cdf822d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:56.044324 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4ee0c356-2aa9-4734-b114-1e24cdf822d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:56.045895 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1365/2736] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:22:55 2022] DELETE /identity/v3/projects/2765f5ff44784088b4652905fa2a5931 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:56.051858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116d675-512d-44d3-a450-0a288c39be51 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:56.052407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116d675-512d-44d3-a450-0a288c39be51 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:56.052407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116d675-512d-44d3-a450-0a288c39be51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:56.071078 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-2116d675-512d-44d3-a450-0a288c39be51 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:56.072384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2116d675-512d-44d3-a450-0a288c39be51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:56.098214 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:56.133412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1372/2737] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:56.141636 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140f37e-3cd8-4b07-860e-1e4f0551c851 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:56.141930 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140f37e-3cd8-4b07-860e-1e4f0551c851 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:56.142209 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140f37e-3cd8-4b07-860e-1e4f0551c851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:56.162360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-1140f37e-3cd8-4b07-860e-1e4f0551c851 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:56.164226 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1140f37e-3cd8-4b07-860e-1e4f0551c851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:56.172814 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1366/2738] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:56.178244 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116d675-512d-44d3-a450-0a288c39be51 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:56.187708 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116d675-512d-44d3-a450-0a288c39be51 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:56.192561 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba71db-72c6-4d10-94b7-7ce0812bf851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:56.193225 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba71db-72c6-4d10-94b7-7ce0812bf851 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:56.193513 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba71db-72c6-4d10-94b7-7ce0812bf851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:56.193794 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba71db-72c6-4d10-94b7-7ce0812bf851 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:56.194652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba71db-72c6-4d10-94b7-7ce0812bf851 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:22:56.195354 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba71db-72c6-4d10-94b7-7ce0812bf851 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:56.197715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba71db-72c6-4d10-94b7-7ce0812bf851 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:56.223661 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1373/2739] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:22:56.227623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140f37e-3cd8-4b07-860e-1e4f0551c851 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:56.236952 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140f37e-3cd8-4b07-860e-1e4f0551c851 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:56.241787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e390ca-aa63-4a34-9d1d-7c6045efb3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:56.242446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e390ca-aa63-4a34-9d1d-7c6045efb3fa admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:56.242761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e390ca-aa63-4a34-9d1d-7c6045efb3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:56.243037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e390ca-aa63-4a34-9d1d-7c6045efb3fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:56.243722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e390ca-aa63-4a34-9d1d-7c6045efb3fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:56.245643 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e390ca-aa63-4a34-9d1d-7c6045efb3fa admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:56.332437 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1367/2740] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:22:56 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:22:56.337483 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba71db-72c6-4d10-94b7-7ce0812bf851 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:56.347853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba71db-72c6-4d10-94b7-7ce0812bf851 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:56.404088 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f4287-c131-43fe-80eb-ff6f155a5ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:56.404750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f4287-c131-43fe-80eb-ff6f155a5ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:56.405034 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f4287-c131-43fe-80eb-ff6f155a5ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:56.405311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f4287-c131-43fe-80eb-ff6f155a5ce1 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:56.406000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f4287-c131-43fe-80eb-ff6f155a5ce1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:56.407969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f4287-c131-43fe-80eb-ff6f155a5ce1 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:56.493143 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1374/2741] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:22:56 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:22:56.497571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e390ca-aa63-4a34-9d1d-7c6045efb3fa admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:56.508563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e390ca-aa63-4a34-9d1d-7c6045efb3fa admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:56.513271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b8039-cb2c-4247-b21c-978a91361f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:56.513983 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b8039-cb2c-4247-b21c-978a91361f50 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:56.514270 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b8039-cb2c-4247-b21c-978a91361f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:56.514570 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b8039-cb2c-4247-b21c-978a91361f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:56.515293 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b8039-cb2c-4247-b21c-978a91361f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:56.516906 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b8039-cb2c-4247-b21c-978a91361f50 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:56.528467 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1368/2742] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:22:56.535459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f4287-c131-43fe-80eb-ff6f155a5ce1 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:56.543854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f4287-c131-43fe-80eb-ff6f155a5ce1 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:56.548656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa980f8a-d507-4abe-ad14-5f8666dc7f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:56.549323 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa980f8a-d507-4abe-ad14-5f8666dc7f3f admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:56.549610 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa980f8a-d507-4abe-ad14-5f8666dc7f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:56.549893 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa980f8a-d507-4abe-ad14-5f8666dc7f3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:56.550666 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa980f8a-d507-4abe-ad14-5f8666dc7f3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:56.552546 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa980f8a-d507-4abe-ad14-5f8666dc7f3f admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:56.563173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1375/2743] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:22:56.567527 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b8039-cb2c-4247-b21c-978a91361f50 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:56.578175 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b8039-cb2c-4247-b21c-978a91361f50 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:56.582846 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d07ca8-cf8d-42ec-8b87-27019ee1bb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:56.583722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d07ca8-cf8d-42ec-8b87-27019ee1bb06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:56.584008 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d07ca8-cf8d-42ec-8b87-27019ee1bb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:56.584278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d07ca8-cf8d-42ec-8b87-27019ee1bb06 admin admin] PATH_INFO: `/v3/projects/5870a8504c4f45849258ad354a8a36f2/users/186bf12ac330466fb9373db24408031e/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:56.600403 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d07ca8-cf8d-42ec-8b87-27019ee1bb06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5870a8504c4f45849258ad354a8a36f2, user_id=186bf12ac330466fb9373db24408031e, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:56.603445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d07ca8-cf8d-42ec-8b87-27019ee1bb06 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:56.674072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1369/2744] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:22:56 2022] PUT /identity/v3/projects/5870a8504c4f45849258ad354a8a36f2/users/186bf12ac330466fb9373db24408031e/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:56.692254 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c7d2d-4926-4440-9a17-41eaa708e1b1 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:56.692552 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c7d2d-4926-4440-9a17-41eaa708e1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:56.692837 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c7d2d-4926-4440-9a17-41eaa708e1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:56.720631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-676c7d2d-4926-4440-9a17-41eaa708e1b1 None None] MFA Rules not processed for user `186bf12ac330466fb9373db24408031e`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:56.722112 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-676c7d2d-4926-4440-9a17-41eaa708e1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:56.787050 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1376/2745] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:56.798380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d07ca8-cf8d-42ec-8b87-27019ee1bb06 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:56.800265 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e7d07ca8-cf8d-42ec-8b87-27019ee1bb06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:56.847870 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:56.863367 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d07ca8-cf8d-42ec-8b87-27019ee1bb06 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:56.870154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80641e-88f7-487f-b6c7-d133c1714d37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:56.870154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80641e-88f7-487f-b6c7-d133c1714d37 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:56.870154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80641e-88f7-487f-b6c7-d133c1714d37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:56.870154 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80641e-88f7-487f-b6c7-d133c1714d37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:56.877113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80641e-88f7-487f-b6c7-d133c1714d37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:56.878604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80641e-88f7-487f-b6c7-d133c1714d37 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:56.888230 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1370/2746] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:22:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:22:56.908535 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c7d2d-4926-4440-9a17-41eaa708e1b1 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:56.941754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:56.979669 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c7d2d-4926-4440-9a17-41eaa708e1b1 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:56.984858 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39115167-b3cd-42c6-a468-5f220981cb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:56.985758 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39115167-b3cd-42c6-a468-5f220981cb24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:56.986423 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39115167-b3cd-42c6-a468-5f220981cb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:56.986902 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39115167-b3cd-42c6-a468-5f220981cb24 admin admin] PATH_INFO: `/v3/users/186bf12ac330466fb9373db24408031e` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:56.988185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39115167-b3cd-42c6-a468-5f220981cb24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=186bf12ac330466fb9373db24408031e)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:56.990132 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39115167-b3cd-42c6-a468-5f220981cb24 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:57.191232 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-39115167-b3cd-42c6-a468-5f220981cb24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '186bf12ac330466fb9373db24408031e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:57.191969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-39115167-b3cd-42c6-a468-5f220981cb24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '186bf12ac330466fb9373db24408031e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:57.192556 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-39115167-b3cd-42c6-a468-5f220981cb24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '186bf12ac330466fb9373db24408031e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:57.338395 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-39115167-b3cd-42c6-a468-5f220981cb24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '186bf12ac330466fb9373db24408031e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:57.352002 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-39115167-b3cd-42c6-a468-5f220981cb24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '186bf12ac330466fb9373db24408031e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:57.364715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1377/2747] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:22:56 2022] DELETE /identity/v3/users/186bf12ac330466fb9373db24408031e => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:57.370058 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cecf09-7110-40ee-8392-d1295c3ec1c5 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:57.370352 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cecf09-7110-40ee-8392-d1295c3ec1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:57.370664 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cecf09-7110-40ee-8392-d1295c3ec1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:57.388351 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-02cecf09-7110-40ee-8392-d1295c3ec1c5 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:57.389534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-02cecf09-7110-40ee-8392-d1295c3ec1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:57.450653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1371/2748] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:57.461149 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39115167-b3cd-42c6-a468-5f220981cb24 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:57.462660 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-39115167-b3cd-42c6-a468-5f220981cb24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:57.519947 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39115167-b3cd-42c6-a468-5f220981cb24 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:57.524198 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efb725-de7c-473a-bb76-40c1c2a24e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:57.524842 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efb725-de7c-473a-bb76-40c1c2a24e14 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:57.525130 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efb725-de7c-473a-bb76-40c1c2a24e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:57.525411 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efb725-de7c-473a-bb76-40c1c2a24e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:57.534351 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8efb725-de7c-473a-bb76-40c1c2a24e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:57.535820 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8efb725-de7c-473a-bb76-40c1c2a24e14 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:57.546341 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1378/2749] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:22:58.013562 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cecf09-7110-40ee-8392-d1295c3ec1c5 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:58.015485 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-02cecf09-7110-40ee-8392-d1295c3ec1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:58.029145 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cecf09-7110-40ee-8392-d1295c3ec1c5 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:58.034364 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5576780-fb8f-49fe-ab89-8b62fc4769d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:58.035498 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5576780-fb8f-49fe-ab89-8b62fc4769d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:58.035780 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5576780-fb8f-49fe-ab89-8b62fc4769d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:58.036060 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5576780-fb8f-49fe-ab89-8b62fc4769d5 admin admin] PATH_INFO: `/v3/projects/5870a8504c4f45849258ad354a8a36f2` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:58.037334 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5576780-fb8f-49fe-ab89-8b62fc4769d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5870a8504c4f45849258ad354a8a36f2)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:58.039901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5576780-fb8f-49fe-ab89-8b62fc4769d5 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:58.133805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f5576780-fb8f-49fe-ab89-8b62fc4769d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5870a8504c4f45849258ad354a8a36f2', 'request_id': 'req-f5576780-fb8f-49fe-ab89-8b62fc4769d5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:58.140310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f5576780-fb8f-49fe-ab89-8b62fc4769d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5870a8504c4f45849258ad354a8a36f2', 'request_id': 'req-f5576780-fb8f-49fe-ab89-8b62fc4769d5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:58.235838 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f5576780-fb8f-49fe-ab89-8b62fc4769d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5870a8504c4f45849258ad354a8a36f2', 'request_id': 'req-f5576780-fb8f-49fe-ab89-8b62fc4769d5'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:58.248050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f5576780-fb8f-49fe-ab89-8b62fc4769d5 admin admin] The token cache is being invalidate because project 5870a8504c4f45849258ad354a8a36f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:22:58.248336 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f5576780-fb8f-49fe-ab89-8b62fc4769d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:58.248984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-f5576780-fb8f-49fe-ab89-8b62fc4769d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:22:58.250563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1372/2750] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:22:58 2022] DELETE /identity/v3/projects/5870a8504c4f45849258ad354a8a36f2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:58.255660 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fa507-0d57-431e-9a2e-e8263815433d None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:58.256171 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fa507-0d57-431e-9a2e-e8263815433d None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:58.256292 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fa507-0d57-431e-9a2e-e8263815433d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:58.275981 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-fb5fa507-0d57-431e-9a2e-e8263815433d None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:58.277225 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-fb5fa507-0d57-431e-9a2e-e8263815433d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:58.320865 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:58.338494 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1379/2751] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:58.405827 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f4f49-e5e8-49b2-a4a6-f1dda57bed5a None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:58.406120 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f4f49-e5e8-49b2-a4a6-f1dda57bed5a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:58.406405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f4f49-e5e8-49b2-a4a6-f1dda57bed5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:58.425909 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-532f4f49-e5e8-49b2-a4a6-f1dda57bed5a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:58.427179 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-532f4f49-e5e8-49b2-a4a6-f1dda57bed5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:58.436212 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1373/2752] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:58.440623 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fa507-0d57-431e-9a2e-e8263815433d None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:58.451234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fa507-0d57-431e-9a2e-e8263815433d None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:58.455891 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc83c1d-17e2-461e-8aed-862521ca852d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:58.456551 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc83c1d-17e2-461e-8aed-862521ca852d admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:58.456840 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc83c1d-17e2-461e-8aed-862521ca852d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:58.457130 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc83c1d-17e2-461e-8aed-862521ca852d admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:58.457857 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc83c1d-17e2-461e-8aed-862521ca852d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:22:58.458170 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc83c1d-17e2-461e-8aed-862521ca852d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:58.459821 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc83c1d-17e2-461e-8aed-862521ca852d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:58.472184 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1380/2753] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:22:58.476543 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f4f49-e5e8-49b2-a4a6-f1dda57bed5a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:58.486132 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f4f49-e5e8-49b2-a4a6-f1dda57bed5a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:58.490862 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9630330-b871-46dd-9b9e-5998a086b2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:58.492037 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9630330-b871-46dd-9b9e-5998a086b2d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:58.492323 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9630330-b871-46dd-9b9e-5998a086b2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:58.492605 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9630330-b871-46dd-9b9e-5998a086b2d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:58.493340 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9630330-b871-46dd-9b9e-5998a086b2d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:58.495322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9630330-b871-46dd-9b9e-5998a086b2d7 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:58.577177 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1374/2754] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:22:58 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:22:58.581412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc83c1d-17e2-461e-8aed-862521ca852d admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:58.592251 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc83c1d-17e2-461e-8aed-862521ca852d admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:58.649756 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075714ad-7e5c-4a0e-a3a6-f66865a323ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:58.650425 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075714ad-7e5c-4a0e-a3a6-f66865a323ca admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:58.650781 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075714ad-7e5c-4a0e-a3a6-f66865a323ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:58.651001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075714ad-7e5c-4a0e-a3a6-f66865a323ca admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:58.651722 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075714ad-7e5c-4a0e-a3a6-f66865a323ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:58.653643 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075714ad-7e5c-4a0e-a3a6-f66865a323ca admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:58.738961 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1381/2755] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:22:58 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:22:58.744562 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9630330-b871-46dd-9b9e-5998a086b2d7 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:58.755256 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9630330-b871-46dd-9b9e-5998a086b2d7 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:58.760237 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e4f86-306e-47b2-8028-eaaf70a06186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:58.760903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e4f86-306e-47b2-8028-eaaf70a06186 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:58.761406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e4f86-306e-47b2-8028-eaaf70a06186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:58.761516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e4f86-306e-47b2-8028-eaaf70a06186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:58.762271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e4f86-306e-47b2-8028-eaaf70a06186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:58.763929 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e4f86-306e-47b2-8028-eaaf70a06186 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:58.774654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1375/2756] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:22:58.778818 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075714ad-7e5c-4a0e-a3a6-f66865a323ca admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:58.789329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075714ad-7e5c-4a0e-a3a6-f66865a323ca admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:58.793988 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b9a53-d36b-438e-84cd-86098eb9e504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:58.794683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b9a53-d36b-438e-84cd-86098eb9e504 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:58.794973 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b9a53-d36b-438e-84cd-86098eb9e504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:58.795258 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b9a53-d36b-438e-84cd-86098eb9e504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:58.795982 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b9a53-d36b-438e-84cd-86098eb9e504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:58.797577 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b9a53-d36b-438e-84cd-86098eb9e504 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:58.808619 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1382/2757] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:22:58.812883 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e4f86-306e-47b2-8028-eaaf70a06186 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:58.823512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e4f86-306e-47b2-8028-eaaf70a06186 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:58.828277 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ffc30c-40fc-405a-b50b-71064cd510b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:58.829163 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffc30c-40fc-405a-b50b-71064cd510b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:58.829451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffc30c-40fc-405a-b50b-71064cd510b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:58.829734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffc30c-40fc-405a-b50b-71064cd510b5 admin admin] PATH_INFO: `/v3/projects/2acd7cd6aee145d2840d0dbd28dadea5/users/ac4127c227664545ac8701c43ea7910e/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:58.844569 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ffc30c-40fc-405a-b50b-71064cd510b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2acd7cd6aee145d2840d0dbd28dadea5, user_id=ac4127c227664545ac8701c43ea7910e, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:58.847566 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ffc30c-40fc-405a-b50b-71064cd510b5 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:58.917606 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:22:58.922881 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1376/2758] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:22:58 2022] PUT /identity/v3/projects/2acd7cd6aee145d2840d0dbd28dadea5/users/ac4127c227664545ac8701c43ea7910e/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:22:58.934369 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51913eb-1529-427b-8f7d-7a5b0c174e42 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:58.934705 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51913eb-1529-427b-8f7d-7a5b0c174e42 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:58.938817 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51913eb-1529-427b-8f7d-7a5b0c174e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:58.961992 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-d51913eb-1529-427b-8f7d-7a5b0c174e42 None None] MFA Rules not processed for user `ac4127c227664545ac8701c43ea7910e`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:22:58.963293 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d51913eb-1529-427b-8f7d-7a5b0c174e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:59.033172 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1383/2759] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:22:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:22:59.045512 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ffc30c-40fc-405a-b50b-71064cd510b5 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:22:59.047269 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a0ffc30c-40fc-405a-b50b-71064cd510b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:22:59.104878 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ffc30c-40fc-405a-b50b-71064cd510b5 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:22:59.108607 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a326a-3f0c-43b4-bbed-7322571a73db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:22:59.109290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a326a-3f0c-43b4-bbed-7322571a73db service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:22:59.109572 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a326a-3f0c-43b4-bbed-7322571a73db service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:22:59.109853 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a326a-3f0c-43b4-bbed-7322571a73db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:22:59.118928 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a326a-3f0c-43b4-bbed-7322571a73db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:22:59.121392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a326a-3f0c-43b4-bbed-7322571a73db service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:22:59.129425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1377/2760] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:22:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:23:09.027756 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:23:09.125268 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=26 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:23:09.443170 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51913eb-1529-427b-8f7d-7a5b0c174e42 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:23:09.453413 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51913eb-1529-427b-8f7d-7a5b0c174e42 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:23:09.456972 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eaaed6-bb15-4418-ba7d-a299d09a49bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:23:09.457606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eaaed6-bb15-4418-ba7d-a299d09a49bf service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:09.457889 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eaaed6-bb15-4418-ba7d-a299d09a49bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:09.458165 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eaaed6-bb15-4418-ba7d-a299d09a49bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:09.460574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e3eaaed6-bb15-4418-ba7d-a299d09a49bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:23:09.519106 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eaaed6-bb15-4418-ba7d-a299d09a49bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:23:09.520556 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eaaed6-bb15-4418-ba7d-a299d09a49bf service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:23:09.531370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1384/2761] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:23:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:23:18.343558 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6e348-6ac3-44b6-b2d3-fa5924afba9d None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:18.344142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6e348-6ac3-44b6-b2d3-fa5924afba9d None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:18.344142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6e348-6ac3-44b6-b2d3-fa5924afba9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:18.363771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-13a6e348-6ac3-44b6-b2d3-fa5924afba9d None None] MFA Rules not processed for user `6bfa2a00688d4eab993059ff176d8920`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:23:18.365093 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-13a6e348-6ac3-44b6-b2d3-fa5924afba9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:23:18.373185 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1378/2762] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:23:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:23:19.526897 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=28 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:23:23.076864 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eaaed6-bb15-4418-ba7d-a299d09a49bf service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:23:23.078442 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e3eaaed6-bb15-4418-ba7d-a299d09a49bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:23:23.142266 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eaaed6-bb15-4418-ba7d-a299d09a49bf service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:23:23.145771 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16c33b-cc43-4a9a-be35-e1e9e7ed9729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:23:23.146485 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16c33b-cc43-4a9a-be35-e1e9e7ed9729 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:23.147971 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16c33b-cc43-4a9a-be35-e1e9e7ed9729 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:23.148352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16c33b-cc43-4a9a-be35-e1e9e7ed9729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:23.150238 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-4a16c33b-cc43-4a9a-be35-e1e9e7ed9729 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:23:23.253381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16c33b-cc43-4a9a-be35-e1e9e7ed9729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:23:23.255692 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16c33b-cc43-4a9a-be35-e1e9e7ed9729 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:23:23.267365 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1385/2763] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:23:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:23:27.697384 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6e348-6ac3-44b6-b2d3-fa5924afba9d None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:23:27.765623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6e348-6ac3-44b6-b2d3-fa5924afba9d None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:23:27.770301 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279642a-6de7-4630-8c3c-680404927beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:23:27.771227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279642a-6de7-4630-8c3c-680404927beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:27.772027 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279642a-6de7-4630-8c3c-680404927beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:27.772336 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279642a-6de7-4630-8c3c-680404927beb admin admin] PATH_INFO: `/v3/users/ac4127c227664545ac8701c43ea7910e` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:27.773661 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279642a-6de7-4630-8c3c-680404927beb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac4127c227664545ac8701c43ea7910e)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:23:27.775745 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279642a-6de7-4630-8c3c-680404927beb admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:23:28.063857 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-d279642a-6de7-4630-8c3c-680404927beb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac4127c227664545ac8701c43ea7910e'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:23:28.072136 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-d279642a-6de7-4630-8c3c-680404927beb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac4127c227664545ac8701c43ea7910e'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:23:28.072781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-d279642a-6de7-4630-8c3c-680404927beb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac4127c227664545ac8701c43ea7910e'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:23:28.089681 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-d279642a-6de7-4630-8c3c-680404927beb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac4127c227664545ac8701c43ea7910e'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:23:28.184127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-d279642a-6de7-4630-8c3c-680404927beb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac4127c227664545ac8701c43ea7910e'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:23:28.190844 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1379/2764] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:23:27 2022] DELETE /identity/v3/users/ac4127c227664545ac8701c43ea7910e => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:23:28.196835 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c236d0-8cba-49de-a270-91949282ec60 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:28.197350 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c236d0-8cba-49de-a270-91949282ec60 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:28.197478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c236d0-8cba-49de-a270-91949282ec60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:28.215888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b3c236d0-8cba-49de-a270-91949282ec60 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:23:28.217141 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b3c236d0-8cba-49de-a270-91949282ec60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:23:28.276439 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1386/2765] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:23:28.286513 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279642a-6de7-4630-8c3c-680404927beb admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:23:28.288201 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d279642a-6de7-4630-8c3c-680404927beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:23:28.350715 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279642a-6de7-4630-8c3c-680404927beb admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:23:28.354790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e74b4-2eca-495f-9ead-caaf834eb784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:23:28.355436 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e74b4-2eca-495f-9ead-caaf834eb784 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:28.355720 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e74b4-2eca-495f-9ead-caaf834eb784 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:28.356026 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e74b4-2eca-495f-9ead-caaf834eb784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:28.366223 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e74b4-2eca-495f-9ead-caaf834eb784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:23:28.367817 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e74b4-2eca-495f-9ead-caaf834eb784 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:23:28.377537 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1380/2766] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:23:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:23:28.804289 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c236d0-8cba-49de-a270-91949282ec60 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:23:28.805887 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b3c236d0-8cba-49de-a270-91949282ec60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:23:28.819316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c236d0-8cba-49de-a270-91949282ec60 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:23:28.823493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb7658-3a1e-4c17-9f34-ec9ff7a0086a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:23:28.824583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb7658-3a1e-4c17-9f34-ec9ff7a0086a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:28.824769 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb7658-3a1e-4c17-9f34-ec9ff7a0086a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:28.825018 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb7658-3a1e-4c17-9f34-ec9ff7a0086a admin admin] PATH_INFO: `/v3/projects/2acd7cd6aee145d2840d0dbd28dadea5` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:28.826271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb7658-3a1e-4c17-9f34-ec9ff7a0086a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2acd7cd6aee145d2840d0dbd28dadea5)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:23:28.828230 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb7658-3a1e-4c17-9f34-ec9ff7a0086a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:23:28.931654 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:23:28.940530 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cecb7658-3a1e-4c17-9f34-ec9ff7a0086a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2acd7cd6aee145d2840d0dbd28dadea5', 'request_id': 'req-cecb7658-3a1e-4c17-9f34-ec9ff7a0086a'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:23:28.947108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cecb7658-3a1e-4c17-9f34-ec9ff7a0086a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2acd7cd6aee145d2840d0dbd28dadea5', 'request_id': 'req-cecb7658-3a1e-4c17-9f34-ec9ff7a0086a'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:23:28.952636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cecb7658-3a1e-4c17-9f34-ec9ff7a0086a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2acd7cd6aee145d2840d0dbd28dadea5', 'request_id': 'req-cecb7658-3a1e-4c17-9f34-ec9ff7a0086a'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:23:29.024500 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cecb7658-3a1e-4c17-9f34-ec9ff7a0086a admin admin] The token cache is being invalidate because project 2acd7cd6aee145d2840d0dbd28dadea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:23:29.024792 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cecb7658-3a1e-4c17-9f34-ec9ff7a0086a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:23:29.025459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-cecb7658-3a1e-4c17-9f34-ec9ff7a0086a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:23:29.027445 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1387/2767] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:23:28 2022] DELETE /identity/v3/projects/2acd7cd6aee145d2840d0dbd28dadea5 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:23:29.032241 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41706f08-53fb-489a-bc3c-61f22b15dd4a None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:29.032752 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41706f08-53fb-489a-bc3c-61f22b15dd4a None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:29.034995 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41706f08-53fb-489a-bc3c-61f22b15dd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:29.051861 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-41706f08-53fb-489a-bc3c-61f22b15dd4a None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:23:29.053997 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-41706f08-53fb-489a-bc3c-61f22b15dd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:23:29.064126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:23:29.118825 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1381/2768] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:23:29.127061 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1811f7-9f05-4387-bc6b-4ea4be85f7e5 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:29.127358 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1811f7-9f05-4387-bc6b-4ea4be85f7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:29.127644 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1811f7-9f05-4387-bc6b-4ea4be85f7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:29.148348 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-9f1811f7-9f05-4387-bc6b-4ea4be85f7e5 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:23:29.149703 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9f1811f7-9f05-4387-bc6b-4ea4be85f7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:23:29.160108 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1388/2769] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:23:29.164859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41706f08-53fb-489a-bc3c-61f22b15dd4a None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:23:29.175532 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41706f08-53fb-489a-bc3c-61f22b15dd4a None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:23:29.183560 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed716d87-5778-4529-bb33-428f79d3d07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:23:29.184392 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed716d87-5778-4529-bb33-428f79d3d07a admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:29.184689 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed716d87-5778-4529-bb33-428f79d3d07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:29.185146 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed716d87-5778-4529-bb33-428f79d3d07a admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:29.185893 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed716d87-5778-4529-bb33-428f79d3d07a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:23:29.186210 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed716d87-5778-4529-bb33-428f79d3d07a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:23:29.187962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed716d87-5778-4529-bb33-428f79d3d07a admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:23:29.201470 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1382/2770] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:23:29.205870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1811f7-9f05-4387-bc6b-4ea4be85f7e5 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:23:29.215563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1811f7-9f05-4387-bc6b-4ea4be85f7e5 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:23:29.220339 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d7dc3-4f7a-4efe-858f-f706f75a43c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:23:29.221188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d7dc3-4f7a-4efe-858f-f706f75a43c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:29.221331 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d7dc3-4f7a-4efe-858f-f706f75a43c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:29.221618 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d7dc3-4f7a-4efe-858f-f706f75a43c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:29.222622 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d7dc3-4f7a-4efe-858f-f706f75a43c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:23:29.224617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d7dc3-4f7a-4efe-858f-f706f75a43c9 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:23:29.316860 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1389/2771] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:23:29 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:23:29.321370 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed716d87-5778-4529-bb33-428f79d3d07a admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:23:29.331874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed716d87-5778-4529-bb33-428f79d3d07a admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:23:29.390560 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc83a5d-3990-4a77-90d2-02f1e6e333c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:23:29.391255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc83a5d-3990-4a77-90d2-02f1e6e333c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:29.391542 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc83a5d-3990-4a77-90d2-02f1e6e333c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:29.391823 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc83a5d-3990-4a77-90d2-02f1e6e333c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:29.392508 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc83a5d-3990-4a77-90d2-02f1e6e333c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:23:29.394466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc83a5d-3990-4a77-90d2-02f1e6e333c6 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:23:29.470121 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1383/2772] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:23:29 2022] POST /identity/v3/users => generated 283 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:23:29.476570 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d7dc3-4f7a-4efe-858f-f706f75a43c9 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:23:29.486925 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d7dc3-4f7a-4efe-858f-f706f75a43c9 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:23:29.492102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d21b1-53bf-4ed6-bddf-d56002e287e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:23:29.492761 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d21b1-53bf-4ed6-bddf-d56002e287e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:29.493046 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d21b1-53bf-4ed6-bddf-d56002e287e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:29.493329 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d21b1-53bf-4ed6-bddf-d56002e287e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:29.494052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d21b1-53bf-4ed6-bddf-d56002e287e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:23:29.495687 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d21b1-53bf-4ed6-bddf-d56002e287e0 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:23:29.507217 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1390/2773] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:23:29.511463 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc83a5d-3990-4a77-90d2-02f1e6e333c6 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:23:29.522625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc83a5d-3990-4a77-90d2-02f1e6e333c6 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:23:29.527446 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e1ffa-d771-44a9-8cc3-68ffe312bfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:23:29.528116 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e1ffa-d771-44a9-8cc3-68ffe312bfd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:29.528413 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e1ffa-d771-44a9-8cc3-68ffe312bfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:29.528717 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e1ffa-d771-44a9-8cc3-68ffe312bfd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:29.529451 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e1ffa-d771-44a9-8cc3-68ffe312bfd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:23:29.531117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e1ffa-d771-44a9-8cc3-68ffe312bfd1 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:23:29.542261 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1384/2774] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:23:29.546810 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d21b1-53bf-4ed6-bddf-d56002e287e0 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:23:29.560162 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d21b1-53bf-4ed6-bddf-d56002e287e0 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:23:29.565227 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473dc4a-0bc5-4b80-b79a-d8eca2bbb2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:23:29.566193 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473dc4a-0bc5-4b80-b79a-d8eca2bbb2e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:29.566517 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473dc4a-0bc5-4b80-b79a-d8eca2bbb2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:29.566677 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473dc4a-0bc5-4b80-b79a-d8eca2bbb2e0 admin admin] PATH_INFO: `/v3/projects/32d80bb7e9904e7586502b4c7f179d99/users/d10006776c4f42b18e7b7770ce7683b8/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:29.583494 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473dc4a-0bc5-4b80-b79a-d8eca2bbb2e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32d80bb7e9904e7586502b4c7f179d99, user_id=d10006776c4f42b18e7b7770ce7683b8, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:23:29.586777 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473dc4a-0bc5-4b80-b79a-d8eca2bbb2e0 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:23:29.662049 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1391/2775] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:23:29 2022] PUT /identity/v3/projects/32d80bb7e9904e7586502b4c7f179d99/users/d10006776c4f42b18e7b7770ce7683b8/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:23:29.672871 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485da35-1156-4fee-bd05-00f1af42be6b None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:29.673194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485da35-1156-4fee-bd05-00f1af42be6b None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:29.673501 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485da35-1156-4fee-bd05-00f1af42be6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:29.703500 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-8485da35-1156-4fee-bd05-00f1af42be6b None None] MFA Rules not processed for user `d10006776c4f42b18e7b7770ce7683b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:23:29.704732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-8485da35-1156-4fee-bd05-00f1af42be6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:23:29.774798 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1385/2776] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:23:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:23:29.787407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473dc4a-0bc5-4b80-b79a-d8eca2bbb2e0 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:23:29.789240 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e473dc4a-0bc5-4b80-b79a-d8eca2bbb2e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:23:29.849086 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473dc4a-0bc5-4b80-b79a-d8eca2bbb2e0 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:23:29.856384 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d9806-9acb-4e81-9b7a-ad678e9d7644 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:23:29.857053 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d9806-9acb-4e81-9b7a-ad678e9d7644 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:29.857339 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d9806-9acb-4e81-9b7a-ad678e9d7644 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:29.857625 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d9806-9acb-4e81-9b7a-ad678e9d7644 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:29.867288 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d9806-9acb-4e81-9b7a-ad678e9d7644 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:23:29.868782 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d9806-9acb-4e81-9b7a-ad678e9d7644 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:23:29.877956 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1392/2777] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:23:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:23:30.328072 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485da35-1156-4fee-bd05-00f1af42be6b None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:23:30.331653 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-8485da35-1156-4fee-bd05-00f1af42be6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:23:30.346471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:23:30.394805 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485da35-1156-4fee-bd05-00f1af42be6b None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:23:30.399560 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dc4fa-3229-4bc4-9abe-9ea073b6fff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:23:30.400143 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dc4fa-3229-4bc4-9abe-9ea073b6fff8 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:30.400428 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dc4fa-3229-4bc4-9abe-9ea073b6fff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:30.400702 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dc4fa-3229-4bc4-9abe-9ea073b6fff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:30.402508 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-400dc4fa-3229-4bc4-9abe-9ea073b6fff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:23:30.475976 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400dc4fa-3229-4bc4-9abe-9ea073b6fff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:23:30.477585 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400dc4fa-3229-4bc4-9abe-9ea073b6fff8 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:23:30.489440 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1386/2778] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:23:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:23:39.873873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=68,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:23:41.315339 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=49 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:23:50.146278 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd1c0b-9303-402f-95d0-59ad87096586 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:23:50.146901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd1c0b-9303-402f-95d0-59ad87096586 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:23:50.146901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd1c0b-9303-402f-95d0-59ad87096586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:23:50.167296 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-acfd1c0b-9303-402f-95d0-59ad87096586 None None] MFA Rules not processed for user `5a9a938a9006499a9426af14cb182ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:23:50.168529 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-acfd1c0b-9303-402f-95d0-59ad87096586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:23:50.227070 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1393/2779] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:23:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:24:00.219249 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=25 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:24:00.277184 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dc4fa-3229-4bc4-9abe-9ea073b6fff8 service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:24:00.286077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dc4fa-3229-4bc4-9abe-9ea073b6fff8 service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:24:00.289804 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980383ee-032d-4757-be49-7626411728fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:24:00.290450 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980383ee-032d-4757-be49-7626411728fd service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:24:00.290786 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980383ee-032d-4757-be49-7626411728fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:24:00.291069 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980383ee-032d-4757-be49-7626411728fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:24:00.292877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-980383ee-032d-4757-be49-7626411728fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:24:00.303555 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980383ee-032d-4757-be49-7626411728fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:24:00.305092 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980383ee-032d-4757-be49-7626411728fd service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:24:00.315414 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1387/2780] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:24:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:24:10.310565 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=6 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:24:10.768019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd1c0b-9303-402f-95d0-59ad87096586 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:24:10.833805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd1c0b-9303-402f-95d0-59ad87096586 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:24:10.838398 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa201871-b63a-481e-b5d3-1966edd8a3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:24:10.839314 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa201871-b63a-481e-b5d3-1966edd8a3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:24:10.839601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa201871-b63a-481e-b5d3-1966edd8a3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:24:10.839879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa201871-b63a-481e-b5d3-1966edd8a3f3 admin admin] PATH_INFO: `/v3/users/d10006776c4f42b18e7b7770ce7683b8` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:24:10.841085 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa201871-b63a-481e-b5d3-1966edd8a3f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d10006776c4f42b18e7b7770ce7683b8)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:24:10.843235 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa201871-b63a-481e-b5d3-1966edd8a3f3 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:24:11.024077 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-fa201871-b63a-481e-b5d3-1966edd8a3f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd10006776c4f42b18e7b7770ce7683b8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:24:11.024644 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-fa201871-b63a-481e-b5d3-1966edd8a3f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd10006776c4f42b18e7b7770ce7683b8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:24:11.025299 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-fa201871-b63a-481e-b5d3-1966edd8a3f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd10006776c4f42b18e7b7770ce7683b8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:24:11.105869 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-fa201871-b63a-481e-b5d3-1966edd8a3f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd10006776c4f42b18e7b7770ce7683b8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:24:11.118870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-fa201871-b63a-481e-b5d3-1966edd8a3f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd10006776c4f42b18e7b7770ce7683b8'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:24:11.131887 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1394/2781] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:24:10 2022] DELETE /identity/v3/users/d10006776c4f42b18e7b7770ce7683b8 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:24:11.136679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e972eee-808f-493b-8cf0-acfe220efc9c None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:24:11.137005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e972eee-808f-493b-8cf0-acfe220efc9c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:24:11.137290 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e972eee-808f-493b-8cf0-acfe220efc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:24:11.156589 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-0e972eee-808f-493b-8cf0-acfe220efc9c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:24:11.157864 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-0e972eee-808f-493b-8cf0-acfe220efc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:24:11.218505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1388/2782] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:24:11.229226 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa201871-b63a-481e-b5d3-1966edd8a3f3 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:24:11.234104 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-fa201871-b63a-481e-b5d3-1966edd8a3f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:24:11.291656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa201871-b63a-481e-b5d3-1966edd8a3f3 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:24:11.295958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33536b49-573f-434f-9d1c-4cc5ac478a29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:24:11.296615 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33536b49-573f-434f-9d1c-4cc5ac478a29 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:24:11.296905 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33536b49-573f-434f-9d1c-4cc5ac478a29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:24:11.297188 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33536b49-573f-434f-9d1c-4cc5ac478a29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:24:11.306349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33536b49-573f-434f-9d1c-4cc5ac478a29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:24:11.307843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33536b49-573f-434f-9d1c-4cc5ac478a29 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:24:11.318059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1395/2783] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:24:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:24:11.759902 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e972eee-808f-493b-8cf0-acfe220efc9c None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:24:11.764265 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-0e972eee-808f-493b-8cf0-acfe220efc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:24:11.774243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e972eee-808f-493b-8cf0-acfe220efc9c None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:24:11.779862 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aeff35-4625-4865-a040-7a946ac28378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:24:11.780914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aeff35-4625-4865-a040-7a946ac28378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:24:11.781204 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aeff35-4625-4865-a040-7a946ac28378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:24:11.781487 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aeff35-4625-4865-a040-7a946ac28378 admin admin] PATH_INFO: `/v3/projects/32d80bb7e9904e7586502b4c7f179d99` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:24:11.782743 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aeff35-4625-4865-a040-7a946ac28378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32d80bb7e9904e7586502b4c7f179d99)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:24:11.784721 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aeff35-4625-4865-a040-7a946ac28378 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:24:11.907701 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e0aeff35-4625-4865-a040-7a946ac28378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32d80bb7e9904e7586502b4c7f179d99', 'request_id': 'req-e0aeff35-4625-4865-a040-7a946ac28378'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:24:11.914355 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e0aeff35-4625-4865-a040-7a946ac28378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32d80bb7e9904e7586502b4c7f179d99', 'request_id': 'req-e0aeff35-4625-4865-a040-7a946ac28378'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:24:11.980233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e0aeff35-4625-4865-a040-7a946ac28378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32d80bb7e9904e7586502b4c7f179d99', 'request_id': 'req-e0aeff35-4625-4865-a040-7a946ac28378'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:24:11.992383 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e0aeff35-4625-4865-a040-7a946ac28378 admin admin] The token cache is being invalidate because project 32d80bb7e9904e7586502b4c7f179d99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:24:11.992671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e0aeff35-4625-4865-a040-7a946ac28378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:24:11.993328 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-e0aeff35-4625-4865-a040-7a946ac28378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:24:11.994939 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1389/2784] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:24:11 2022] DELETE /identity/v3/projects/32d80bb7e9904e7586502b4c7f179d99 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:24:21.313786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:24:21.989118 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:24:52.022968 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33536b49-573f-434f-9d1c-4cc5ac478a29 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:24:52.025616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-33536b49-573f-434f-9d1c-4cc5ac478a29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:24:52.117455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33536b49-573f-434f-9d1c-4cc5ac478a29 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:24:52.203392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fa7fc-8da8-4fcb-80ac-01efa6194478 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:24:52.203960 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fa7fc-8da8-4fcb-80ac-01efa6194478 service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:24:52.204356 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fa7fc-8da8-4fcb-80ac-01efa6194478 service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:24:52.204641 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fa7fc-8da8-4fcb-80ac-01efa6194478 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:24:52.206774 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-233fa7fc-8da8-4fcb-80ac-01efa6194478 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:24:52.220540 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fa7fc-8da8-4fcb-80ac-01efa6194478 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:24:52.223167 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fa7fc-8da8-4fcb-80ac-01efa6194478 service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:24:52.236767 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1396/2785] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:24:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:24:57.597054 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d7cea-a476-42e3-9655-aad38d9c4bf9 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:24:57.597711 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d7cea-a476-42e3-9655-aad38d9c4bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:24:57.597711 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d7cea-a476-42e3-9655-aad38d9c4bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:24:57.616441 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-189d7cea-a476-42e3-9655-aad38d9c4bf9 None None] MFA Rules not processed for user `5a9a938a9006499a9426af14cb182ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:24:57.617720 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-189d7cea-a476-42e3-9655-aad38d9c4bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:24:57.675140 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1390/2786] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:24:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:24:57.989741 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fa7fc-8da8-4fcb-80ac-01efa6194478 service nova] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:24:57.991284 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-233fa7fc-8da8-4fcb-80ac-01efa6194478 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:24:58.054204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fa7fc-8da8-4fcb-80ac-01efa6194478 service nova] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:24:58.057750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe0b9d-0968-499c-ae95-aabed4882f22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:24:58.059042 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe0b9d-0968-499c-ae95-aabed4882f22 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:24:58.059327 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe0b9d-0968-499c-ae95-aabed4882f22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:24:58.059607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe0b9d-0968-499c-ae95-aabed4882f22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:24:58.061427 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9ffe0b9d-0968-499c-ae95-aabed4882f22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:24:58.071264 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe0b9d-0968-499c-ae95-aabed4882f22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:24:58.072694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe0b9d-0968-499c-ae95-aabed4882f22 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:24:58.083053 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1397/2787] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:24:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:24:58.513202 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d7cea-a476-42e3-9655-aad38d9c4bf9 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:24:58.514735 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-189d7cea-a476-42e3-9655-aad38d9c4bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:24:58.587869 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d7cea-a476-42e3-9655-aad38d9c4bf9 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:24:58.593768 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20fd9f-eb1e-41d1-8768-051bf26d9197 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76cc0ae48b5a49c6a7a1c11a08c389e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:24:58.594433 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20fd9f-eb1e-41d1-8768-051bf26d9197 service placement] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:24:58.594762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20fd9f-eb1e-41d1-8768-051bf26d9197 service placement] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:24:58.595046 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20fd9f-eb1e-41d1-8768-051bf26d9197 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:24:58.597142 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ad20fd9f-eb1e-41d1-8768-051bf26d9197 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:24:58.609615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20fd9f-eb1e-41d1-8768-051bf26d9197 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:24:58.611309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20fd9f-eb1e-41d1-8768-051bf26d9197 service placement] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:24:58.622148 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1391/2788] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:24:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:25:06.154200 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe0b9d-0968-499c-ae95-aabed4882f22 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:25:06.156495 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-9ffe0b9d-0968-499c-ae95-aabed4882f22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:25:06.176082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe0b9d-0968-499c-ae95-aabed4882f22 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:25:06.181742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cdde9-2134-4fb0-8b96-344ef6e2b11c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:25:06.182396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cdde9-2134-4fb0-8b96-344ef6e2b11c service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:25:06.182747 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cdde9-2134-4fb0-8b96-344ef6e2b11c service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:25:06.183599 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cdde9-2134-4fb0-8b96-344ef6e2b11c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:25:06.185623 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2b0cdde9-2134-4fb0-8b96-344ef6e2b11c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:25:06.199604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0cdde9-2134-4fb0-8b96-344ef6e2b11c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:25:06.201275 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0cdde9-2134-4fb0-8b96-344ef6e2b11c service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:25:06.212574 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1398/2789] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:25:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:25:08.617107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=57 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:25:10.254905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20fd9f-eb1e-41d1-8768-051bf26d9197 service placement] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:25:10.256499 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-ad20fd9f-eb1e-41d1-8768-051bf26d9197 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:25:10.272685 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20fd9f-eb1e-41d1-8768-051bf26d9197 service placement] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:25:10.279347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44cd1d-4dad-41c2-b0af-f5fc684a6d5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:25:10.280117 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44cd1d-4dad-41c2-b0af-f5fc684a6d5f service nova] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:25:10.280478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44cd1d-4dad-41c2-b0af-f5fc684a6d5f service nova] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:25:10.280855 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44cd1d-4dad-41c2-b0af-f5fc684a6d5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:25:10.282969 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-8a44cd1d-4dad-41c2-b0af-f5fc684a6d5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:25:10.296949 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44cd1d-4dad-41c2-b0af-f5fc684a6d5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:25:10.298711 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44cd1d-4dad-41c2-b0af-f5fc684a6d5f service nova] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:25:10.309899 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1392/2790] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:25:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:25:16.207949 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=94 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:25:20.305168 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=6 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:25:37.326199 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cdde9-2134-4fb0-8b96-344ef6e2b11c service nova] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:25:37.327694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2b0cdde9-2134-4fb0-8b96-344ef6e2b11c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:25:37.339969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cdde9-2134-4fb0-8b96-344ef6e2b11c service nova] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:25:37.343604 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680d141-9333-4dab-b217-4580ca5bec50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:25:37.344252 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680d141-9333-4dab-b217-4580ca5bec50 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:25:37.344542 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680d141-9333-4dab-b217-4580ca5bec50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:25:37.344824 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680d141-9333-4dab-b217-4580ca5bec50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:25:37.346883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a680d141-9333-4dab-b217-4580ca5bec50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:25:37.412816 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680d141-9333-4dab-b217-4580ca5bec50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:25:37.414277 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680d141-9333-4dab-b217-4580ca5bec50 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:25:37.424867 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1399/2791] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:25:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:25:40.913556 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44cd1d-4dad-41c2-b0af-f5fc684a6d5f service nova] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:25:40.915179 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-8a44cd1d-4dad-41c2-b0af-f5fc684a6d5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:25:40.934273 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44cd1d-4dad-41c2-b0af-f5fc684a6d5f service nova] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:25:40.938156 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405b37c-11a7-4f3d-ad06-8f6c844500cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76cc0ae48b5a49c6a7a1c11a08c389e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:25:40.938859 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405b37c-11a7-4f3d-ad06-8f6c844500cb service placement] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:25:40.939149 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405b37c-11a7-4f3d-ad06-8f6c844500cb service placement] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:25:40.939425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405b37c-11a7-4f3d-ad06-8f6c844500cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:25:40.941267 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-6405b37c-11a7-4f3d-ad06-8f6c844500cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:25:40.954301 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405b37c-11a7-4f3d-ad06-8f6c844500cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:25:40.955995 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405b37c-11a7-4f3d-ad06-8f6c844500cb service placement] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:25:40.968133 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1393/2792] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:25:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:25:44.940005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680d141-9333-4dab-b217-4580ca5bec50 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:25:44.941526 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a680d141-9333-4dab-b217-4580ca5bec50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:25:44.958652 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680d141-9333-4dab-b217-4580ca5bec50 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:25:44.961370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c8f87-1140-4448-a0d2-83bdce231556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:25:44.962423 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c8f87-1140-4448-a0d2-83bdce231556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:25:44.963440 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c8f87-1140-4448-a0d2-83bdce231556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:25:44.963931 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c8f87-1140-4448-a0d2-83bdce231556 admin admin] PATH_INFO: `/v3/users/6bfa2a00688d4eab993059ff176d8920` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:25:44.965140 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c8f87-1140-4448-a0d2-83bdce231556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bfa2a00688d4eab993059ff176d8920)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:25:44.967096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c8f87-1140-4448-a0d2-83bdce231556 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:25:45.195744 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a29c8f87-1140-4448-a0d2-83bdce231556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bfa2a00688d4eab993059ff176d8920'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:45.196653 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a29c8f87-1140-4448-a0d2-83bdce231556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bfa2a00688d4eab993059ff176d8920'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:45.197489 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a29c8f87-1140-4448-a0d2-83bdce231556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bfa2a00688d4eab993059ff176d8920'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:45.292799 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a29c8f87-1140-4448-a0d2-83bdce231556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bfa2a00688d4eab993059ff176d8920'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:45.305818 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-a29c8f87-1140-4448-a0d2-83bdce231556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bfa2a00688d4eab993059ff176d8920'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:45.318656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1400/2793] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:25:44 2022] DELETE /identity/v3/users/6bfa2a00688d4eab993059ff176d8920 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:25:45.577812 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405b37c-11a7-4f3d-ad06-8f6c844500cb service placement] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:25:45.590140 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405b37c-11a7-4f3d-ad06-8f6c844500cb service placement] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:25:45.651094 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea12d7b-b609-4453-8386-f3d499e95bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:25:45.651966 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea12d7b-b609-4453-8386-f3d499e95bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:25:45.652251 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea12d7b-b609-4453-8386-f3d499e95bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:25:45.652530 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea12d7b-b609-4453-8386-f3d499e95bc7 admin admin] PATH_INFO: `/v3/projects/94513773e0c94187bbdaeb82c7304eae` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:25:45.653754 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea12d7b-b609-4453-8386-f3d499e95bc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94513773e0c94187bbdaeb82c7304eae)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:25:45.655728 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea12d7b-b609-4453-8386-f3d499e95bc7 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:25:45.801872 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-aea12d7b-b609-4453-8386-f3d499e95bc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94513773e0c94187bbdaeb82c7304eae', 'request_id': 'req-aea12d7b-b609-4453-8386-f3d499e95bc7'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:45.808379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-aea12d7b-b609-4453-8386-f3d499e95bc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94513773e0c94187bbdaeb82c7304eae', 'request_id': 'req-aea12d7b-b609-4453-8386-f3d499e95bc7'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:45.886762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-aea12d7b-b609-4453-8386-f3d499e95bc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94513773e0c94187bbdaeb82c7304eae', 'request_id': 'req-aea12d7b-b609-4453-8386-f3d499e95bc7'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:45.898033 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-aea12d7b-b609-4453-8386-f3d499e95bc7 admin admin] The token cache is being invalidate because project 94513773e0c94187bbdaeb82c7304eae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:25:45.898325 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-aea12d7b-b609-4453-8386-f3d499e95bc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:45.899011 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-aea12d7b-b609-4453-8386-f3d499e95bc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:45.901198 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1394/2794] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:25:45 2022] DELETE /identity/v3/projects/94513773e0c94187bbdaeb82c7304eae => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:25:45.905455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c8f87-1140-4448-a0d2-83bdce231556 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:25:45.906965 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a29c8f87-1140-4448-a0d2-83bdce231556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:25:45.920983 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c8f87-1140-4448-a0d2-83bdce231556 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:25:45.980563 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035140c3-7baf-4914-83e5-0000e02ca118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:25:45.981455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035140c3-7baf-4914-83e5-0000e02ca118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:25:45.981743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035140c3-7baf-4914-83e5-0000e02ca118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:25:45.982024 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035140c3-7baf-4914-83e5-0000e02ca118 admin admin] PATH_INFO: `/v3/users/eaf034a79b434d0a9f79c76b0b8fcc2c` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:25:45.983307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035140c3-7baf-4914-83e5-0000e02ca118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaf034a79b434d0a9f79c76b0b8fcc2c)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:25:45.985263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035140c3-7baf-4914-83e5-0000e02ca118 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:25:45.991001 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:25:46.175175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-035140c3-7baf-4914-83e5-0000e02ca118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaf034a79b434d0a9f79c76b0b8fcc2c'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:46.175838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-035140c3-7baf-4914-83e5-0000e02ca118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaf034a79b434d0a9f79c76b0b8fcc2c'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:46.176465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-035140c3-7baf-4914-83e5-0000e02ca118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaf034a79b434d0a9f79c76b0b8fcc2c'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:46.304511 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-035140c3-7baf-4914-83e5-0000e02ca118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaf034a79b434d0a9f79c76b0b8fcc2c'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:46.318013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-035140c3-7baf-4914-83e5-0000e02ca118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaf034a79b434d0a9f79c76b0b8fcc2c'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:46.331581 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1401/2795] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:25:45 2022] DELETE /identity/v3/users/eaf034a79b434d0a9f79c76b0b8fcc2c => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:25:46.620338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea12d7b-b609-4453-8386-f3d499e95bc7 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:25:46.620338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-aea12d7b-b609-4453-8386-f3d499e95bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:25:46.635981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea12d7b-b609-4453-8386-f3d499e95bc7 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:25:46.723461 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ce0cd-b6c3-4f9d-b569-875aa761af2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:25:46.724779 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ce0cd-b6c3-4f9d-b569-875aa761af2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:25:46.725335 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ce0cd-b6c3-4f9d-b569-875aa761af2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:25:46.725809 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ce0cd-b6c3-4f9d-b569-875aa761af2d admin admin] PATH_INFO: `/v3/projects/57feb941d284478fa86bbc3166240b61` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:25:46.727216 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ce0cd-b6c3-4f9d-b569-875aa761af2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=57feb941d284478fa86bbc3166240b61)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:25:46.729507 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ce0cd-b6c3-4f9d-b569-875aa761af2d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:25:46.755916 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:25:46.859687 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-638ce0cd-b6c3-4f9d-b569-875aa761af2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57feb941d284478fa86bbc3166240b61', 'request_id': 'req-638ce0cd-b6c3-4f9d-b569-875aa761af2d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:46.866208 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-638ce0cd-b6c3-4f9d-b569-875aa761af2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57feb941d284478fa86bbc3166240b61', 'request_id': 'req-638ce0cd-b6c3-4f9d-b569-875aa761af2d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:46.947322 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-638ce0cd-b6c3-4f9d-b569-875aa761af2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57feb941d284478fa86bbc3166240b61', 'request_id': 'req-638ce0cd-b6c3-4f9d-b569-875aa761af2d'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:46.959104 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-638ce0cd-b6c3-4f9d-b569-875aa761af2d admin admin] The token cache is being invalidate because project 57feb941d284478fa86bbc3166240b61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:25:46.961338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-638ce0cd-b6c3-4f9d-b569-875aa761af2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:46.961338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-638ce0cd-b6c3-4f9d-b569-875aa761af2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:25:46.961649 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1395/2796] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:25:46 2022] DELETE /identity/v3/projects/57feb941d284478fa86bbc3166240b61 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:25:56.443638 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:25:56.958312 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:09.759518 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035140c3-7baf-4914-83e5-0000e02ca118 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:09.761478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-035140c3-7baf-4914-83e5-0000e02ca118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:09.827740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035140c3-7baf-4914-83e5-0000e02ca118 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:09.905175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4370d-0ba4-4d35-ba16-7fcfdd0b6888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:09.905175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4370d-0ba4-4d35-ba16-7fcfdd0b6888 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:09.905175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4370d-0ba4-4d35-ba16-7fcfdd0b6888 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:09.905175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4370d-0ba4-4d35-ba16-7fcfdd0b6888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:09.905997 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-d9b4370d-0ba4-4d35-ba16-7fcfdd0b6888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:09.989082 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4370d-0ba4-4d35-ba16-7fcfdd0b6888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:09.990712 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4370d-0ba4-4d35-ba16-7fcfdd0b6888 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:10.002463 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1402/2797] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:26:10.639469 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ce0cd-b6c3-4f9d-b569-875aa761af2d admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:10.641134 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-638ce0cd-b6c3-4f9d-b569-875aa761af2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:10.655608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ce0cd-b6c3-4f9d-b569-875aa761af2d admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:10.660541 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6150dba-6aff-42f9-83e6-96de356c34cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:10.661425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6150dba-6aff-42f9-83e6-96de356c34cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:10.661713 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6150dba-6aff-42f9-83e6-96de356c34cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:10.661995 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6150dba-6aff-42f9-83e6-96de356c34cf admin admin] PATH_INFO: `/v3/users/5a9a938a9006499a9426af14cb182ecc` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:10.663261 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6150dba-6aff-42f9-83e6-96de356c34cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a9a938a9006499a9426af14cb182ecc)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:10.665967 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6150dba-6aff-42f9-83e6-96de356c34cf admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:10.870713 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b6150dba-6aff-42f9-83e6-96de356c34cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a9a938a9006499a9426af14cb182ecc'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:10.877553 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b6150dba-6aff-42f9-83e6-96de356c34cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a9a938a9006499a9426af14cb182ecc'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:10.878229 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b6150dba-6aff-42f9-83e6-96de356c34cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a9a938a9006499a9426af14cb182ecc'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:10.890516 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b6150dba-6aff-42f9-83e6-96de356c34cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a9a938a9006499a9426af14cb182ecc'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:10.959957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b6150dba-6aff-42f9-83e6-96de356c34cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a9a938a9006499a9426af14cb182ecc'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:10.966874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1396/2798] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:26:10 2022] DELETE /identity/v3/users/5a9a938a9006499a9426af14cb182ecc => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:11.265321 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4370d-0ba4-4d35-ba16-7fcfdd0b6888 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:11.277285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4370d-0ba4-4d35-ba16-7fcfdd0b6888 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:11.337223 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73016ac-f47f-4789-88fe-5cdd2b609b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:11.338118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73016ac-f47f-4789-88fe-5cdd2b609b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:11.338409 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73016ac-f47f-4789-88fe-5cdd2b609b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:11.338756 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73016ac-f47f-4789-88fe-5cdd2b609b3e admin admin] PATH_INFO: `/v3/projects/558c29b107e14e3f9b2aa041a388dfa4` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:11.339958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73016ac-f47f-4789-88fe-5cdd2b609b3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=558c29b107e14e3f9b2aa041a388dfa4)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:11.341883 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73016ac-f47f-4789-88fe-5cdd2b609b3e admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:11.350525 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=100 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:11.426063 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b73016ac-f47f-4789-88fe-5cdd2b609b3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '558c29b107e14e3f9b2aa041a388dfa4', 'request_id': 'req-b73016ac-f47f-4789-88fe-5cdd2b609b3e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:11.432291 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b73016ac-f47f-4789-88fe-5cdd2b609b3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '558c29b107e14e3f9b2aa041a388dfa4', 'request_id': 'req-b73016ac-f47f-4789-88fe-5cdd2b609b3e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:11.438131 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b73016ac-f47f-4789-88fe-5cdd2b609b3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '558c29b107e14e3f9b2aa041a388dfa4', 'request_id': 'req-b73016ac-f47f-4789-88fe-5cdd2b609b3e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:11.511764 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b73016ac-f47f-4789-88fe-5cdd2b609b3e admin admin] The token cache is being invalidate because project 558c29b107e14e3f9b2aa041a388dfa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:26:11.512068 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b73016ac-f47f-4789-88fe-5cdd2b609b3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:11.512728 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b73016ac-f47f-4789-88fe-5cdd2b609b3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:11.514301 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1403/2799] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:26:11 2022] DELETE /identity/v3/projects/558c29b107e14e3f9b2aa041a388dfa4 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:11.519697 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d395c-3a71-476b-9c66-7e89bd505eac None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:11.519815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d395c-3a71-476b-9c66-7e89bd505eac None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:11.520057 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d395c-3a71-476b-9c66-7e89bd505eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:11.540274 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-d67d395c-3a71-476b-9c66-7e89bd505eac None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:11.541510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d67d395c-3a71-476b-9c66-7e89bd505eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:11.609739 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1397/2800] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:11.619278 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30c8cc-7ea7-4035-b0f7-072faf9ce0b9 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:11.619580 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30c8cc-7ea7-4035-b0f7-072faf9ce0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:11.619856 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30c8cc-7ea7-4035-b0f7-072faf9ce0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:11.638894 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-bc30c8cc-7ea7-4035-b0f7-072faf9ce0b9 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:11.640132 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-bc30c8cc-7ea7-4035-b0f7-072faf9ce0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:11.648180 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1404/2801] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:11.652746 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d395c-3a71-476b-9c66-7e89bd505eac None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:11.662591 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d395c-3a71-476b-9c66-7e89bd505eac None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:11.667536 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae7d0f-facf-4523-8f7a-94b0c64d4e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:11.668198 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae7d0f-facf-4523-8f7a-94b0c64d4e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:11.668483 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae7d0f-facf-4523-8f7a-94b0c64d4e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:11.668762 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae7d0f-facf-4523-8f7a-94b0c64d4e5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:11.669494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae7d0f-facf-4523-8f7a-94b0c64d4e5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:26:11.669801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae7d0f-facf-4523-8f7a-94b0c64d4e5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:11.671460 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae7d0f-facf-4523-8f7a-94b0c64d4e5a admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:11.684962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1398/2802] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:26:11.689590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30c8cc-7ea7-4035-b0f7-072faf9ce0b9 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:11.699764 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30c8cc-7ea7-4035-b0f7-072faf9ce0b9 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:11.704493 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7f4f8-dc7a-48da-9191-017c4fa7f6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:11.705146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7f4f8-dc7a-48da-9191-017c4fa7f6ff admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:11.705435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7f4f8-dc7a-48da-9191-017c4fa7f6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:11.705719 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7f4f8-dc7a-48da-9191-017c4fa7f6ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:11.706432 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf7f4f8-dc7a-48da-9191-017c4fa7f6ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:11.708377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf7f4f8-dc7a-48da-9191-017c4fa7f6ff admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:11.817116 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1405/2803] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:26:11 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:26:11.822719 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae7d0f-facf-4523-8f7a-94b0c64d4e5a admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:11.833243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae7d0f-facf-4523-8f7a-94b0c64d4e5a admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:11.902301 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ea792-14f5-4402-b85a-f04bc29da13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:11.902994 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ea792-14f5-4402-b85a-f04bc29da13a admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:11.903276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ea792-14f5-4402-b85a-f04bc29da13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:11.903550 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ea792-14f5-4402-b85a-f04bc29da13a admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:11.904233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ea792-14f5-4402-b85a-f04bc29da13a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:11.906187 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ea792-14f5-4402-b85a-f04bc29da13a admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:11.912767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:12.003692 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1399/2804] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:26:11 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:26:12.008236 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7f4f8-dc7a-48da-9191-017c4fa7f6ff admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:12.019017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7f4f8-dc7a-48da-9191-017c4fa7f6ff admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:12.024175 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d9354-0fcf-4f95-a6ef-63fd48d3ddca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:12.024873 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d9354-0fcf-4f95-a6ef-63fd48d3ddca admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:12.025149 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d9354-0fcf-4f95-a6ef-63fd48d3ddca admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:12.025450 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d9354-0fcf-4f95-a6ef-63fd48d3ddca admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:12.026181 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d9354-0fcf-4f95-a6ef-63fd48d3ddca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:12.027925 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d9354-0fcf-4f95-a6ef-63fd48d3ddca admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:12.038721 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1406/2805] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:26:12.043716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ea792-14f5-4402-b85a-f04bc29da13a admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:12.054051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ea792-14f5-4402-b85a-f04bc29da13a admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:12.059989 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f791d-566e-43b1-9650-bac837ede890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:12.060716 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f791d-566e-43b1-9650-bac837ede890 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:12.061030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f791d-566e-43b1-9650-bac837ede890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:12.061310 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f791d-566e-43b1-9650-bac837ede890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:12.062043 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f791d-566e-43b1-9650-bac837ede890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:12.063712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f791d-566e-43b1-9650-bac837ede890 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:12.075051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1400/2806] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:26:12.079250 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d9354-0fcf-4f95-a6ef-63fd48d3ddca admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:12.089888 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d9354-0fcf-4f95-a6ef-63fd48d3ddca admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:12.094631 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0c08d-3e9b-4ce6-b008-625cfe25b023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:12.095515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0c08d-3e9b-4ce6-b008-625cfe25b023 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:12.095796 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0c08d-3e9b-4ce6-b008-625cfe25b023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:12.096330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0c08d-3e9b-4ce6-b008-625cfe25b023 admin admin] PATH_INFO: `/v3/projects/c0ecebd983324ead82b4fe7a86583e3a/users/cd1a25c255244bbca89a0dc8689b1915/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:12.111742 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0c08d-3e9b-4ce6-b008-625cfe25b023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ecebd983324ead82b4fe7a86583e3a, user_id=cd1a25c255244bbca89a0dc8689b1915, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:12.114855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0c08d-3e9b-4ce6-b008-625cfe25b023 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:12.191459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1407/2807] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:26:12 2022] PUT /identity/v3/projects/c0ecebd983324ead82b4fe7a86583e3a/users/cd1a25c255244bbca89a0dc8689b1915/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:12.202406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b79f2-09eb-4b12-bcae-83173fb4689c None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:12.202737 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b79f2-09eb-4b12-bcae-83173fb4689c None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:12.203065 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b79f2-09eb-4b12-bcae-83173fb4689c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:12.230597 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-0b7b79f2-09eb-4b12-bcae-83173fb4689c None None] MFA Rules not processed for user `cd1a25c255244bbca89a0dc8689b1915`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:12.231860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-0b7b79f2-09eb-4b12-bcae-83173fb4689c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:12.297764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1401/2808] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:12.310233 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0c08d-3e9b-4ce6-b008-625cfe25b023 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:12.311994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-73a0c08d-3e9b-4ce6-b008-625cfe25b023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:12.386854 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0c08d-3e9b-4ce6-b008-625cfe25b023 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:12.392668 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04273d7a-3a8f-4504-bf40-1164860ef60c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e1355d18fb049468651b3675c217d60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:12.393318 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04273d7a-3a8f-4504-bf40-1164860ef60c service nova] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:12.393638 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04273d7a-3a8f-4504-bf40-1164860ef60c service nova] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:12.393913 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04273d7a-3a8f-4504-bf40-1164860ef60c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:12.401947 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04273d7a-3a8f-4504-bf40-1164860ef60c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:12.403607 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04273d7a-3a8f-4504-bf40-1164860ef60c service nova] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:12.412747 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1408/2809] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:26:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:26:12.653739 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b79f2-09eb-4b12-bcae-83173fb4689c None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:12.718693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b79f2-09eb-4b12-bcae-83173fb4689c None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:12.723494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18162cc-e2bc-4c7d-b705-903753573484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:12.724363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18162cc-e2bc-4c7d-b705-903753573484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:12.724647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18162cc-e2bc-4c7d-b705-903753573484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:12.724925 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18162cc-e2bc-4c7d-b705-903753573484 admin admin] PATH_INFO: `/v3/users/cd1a25c255244bbca89a0dc8689b1915` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:12.726115 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18162cc-e2bc-4c7d-b705-903753573484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd1a25c255244bbca89a0dc8689b1915)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:12.728625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18162cc-e2bc-4c7d-b705-903753573484 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:13.075146 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a18162cc-e2bc-4c7d-b705-903753573484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd1a25c255244bbca89a0dc8689b1915'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:13.082336 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a18162cc-e2bc-4c7d-b705-903753573484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd1a25c255244bbca89a0dc8689b1915'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:13.083078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a18162cc-e2bc-4c7d-b705-903753573484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd1a25c255244bbca89a0dc8689b1915'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:13.094196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:13.096815 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a18162cc-e2bc-4c7d-b705-903753573484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd1a25c255244bbca89a0dc8689b1915'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:13.203030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a18162cc-e2bc-4c7d-b705-903753573484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd1a25c255244bbca89a0dc8689b1915'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:13.209630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1402/2810] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:26:12 2022] DELETE /identity/v3/users/cd1a25c255244bbca89a0dc8689b1915 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:13.215104 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d43f3-0fae-4ffb-b5af-d8a387cf6bec None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:13.215410 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d43f3-0fae-4ffb-b5af-d8a387cf6bec None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:13.215682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d43f3-0fae-4ffb-b5af-d8a387cf6bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:13.234466 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-318d43f3-0fae-4ffb-b5af-d8a387cf6bec None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:13.235859 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-318d43f3-0fae-4ffb-b5af-d8a387cf6bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:13.249169 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:13.300033 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1409/2811] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:13.310422 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18162cc-e2bc-4c7d-b705-903753573484 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:13.311868 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a18162cc-e2bc-4c7d-b705-903753573484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:13.375763 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18162cc-e2bc-4c7d-b705-903753573484 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:13.379404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b49edd-b0ed-49e3-bd42-a3d477e96a69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:13.380063 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b49edd-b0ed-49e3-bd42-a3d477e96a69 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:13.380346 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b49edd-b0ed-49e3-bd42-a3d477e96a69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:13.380630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b49edd-b0ed-49e3-bd42-a3d477e96a69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:13.390550 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b49edd-b0ed-49e3-bd42-a3d477e96a69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:13.392049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b49edd-b0ed-49e3-bd42-a3d477e96a69 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:13.403188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1403/2812] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:26:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:26:13.893457 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d43f3-0fae-4ffb-b5af-d8a387cf6bec None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:13.895709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-318d43f3-0fae-4ffb-b5af-d8a387cf6bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:13.908983 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d43f3-0fae-4ffb-b5af-d8a387cf6bec None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:13.913715 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e74183-cb28-43d5-8204-997ef08145e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:13.915416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e74183-cb28-43d5-8204-997ef08145e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:13.915707 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e74183-cb28-43d5-8204-997ef08145e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:13.915987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e74183-cb28-43d5-8204-997ef08145e3 admin admin] PATH_INFO: `/v3/projects/c0ecebd983324ead82b4fe7a86583e3a` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:13.917239 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e74183-cb28-43d5-8204-997ef08145e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ecebd983324ead82b4fe7a86583e3a)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:13.919332 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e74183-cb28-43d5-8204-997ef08145e3 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:14.033773 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-48e74183-cb28-43d5-8204-997ef08145e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ecebd983324ead82b4fe7a86583e3a', 'request_id': 'req-48e74183-cb28-43d5-8204-997ef08145e3'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:14.040884 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-48e74183-cb28-43d5-8204-997ef08145e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ecebd983324ead82b4fe7a86583e3a', 'request_id': 'req-48e74183-cb28-43d5-8204-997ef08145e3'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:14.046839 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-48e74183-cb28-43d5-8204-997ef08145e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ecebd983324ead82b4fe7a86583e3a', 'request_id': 'req-48e74183-cb28-43d5-8204-997ef08145e3'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:14.125119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-48e74183-cb28-43d5-8204-997ef08145e3 admin admin] The token cache is being invalidate because project c0ecebd983324ead82b4fe7a86583e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:26:14.125409 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-48e74183-cb28-43d5-8204-997ef08145e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:14.126052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-48e74183-cb28-43d5-8204-997ef08145e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:14.127668 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1410/2813] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:26:13 2022] DELETE /identity/v3/projects/c0ecebd983324ead82b4fe7a86583e3a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:14.132853 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c40858-03c2-4ca8-b229-c69c56d986ae None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:14.133857 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c40858-03c2-4ca8-b229-c69c56d986ae None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:14.133959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c40858-03c2-4ca8-b229-c69c56d986ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:14.152806 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-82c40858-03c2-4ca8-b229-c69c56d986ae None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:14.153048 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-82c40858-03c2-4ca8-b229-c69c56d986ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:14.212081 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1404/2814] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:14.220441 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ad8f0-2545-4f82-b0f5-f604860c6f56 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:14.220735 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ad8f0-2545-4f82-b0f5-f604860c6f56 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:14.221289 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ad8f0-2545-4f82-b0f5-f604860c6f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:14.240211 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-961ad8f0-2545-4f82-b0f5-f604860c6f56 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:14.241439 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-961ad8f0-2545-4f82-b0f5-f604860c6f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:14.250215 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1411/2815] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:14.254764 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c40858-03c2-4ca8-b229-c69c56d986ae None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:14.264536 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c40858-03c2-4ca8-b229-c69c56d986ae None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:14.270064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90c3c0-903c-4a96-9129-4de9faca2861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:14.270761 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90c3c0-903c-4a96-9129-4de9faca2861 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:14.271045 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90c3c0-903c-4a96-9129-4de9faca2861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:14.271319 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90c3c0-903c-4a96-9129-4de9faca2861 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:14.272051 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa90c3c0-903c-4a96-9129-4de9faca2861 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:26:14.272354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa90c3c0-903c-4a96-9129-4de9faca2861 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:14.274158 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa90c3c0-903c-4a96-9129-4de9faca2861 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:14.286656 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1405/2816] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:26:14.292407 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ad8f0-2545-4f82-b0f5-f604860c6f56 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:14.301887 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ad8f0-2545-4f82-b0f5-f604860c6f56 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:14.307287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e766e-644f-4c16-9b26-60a134d28724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:14.307944 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e766e-644f-4c16-9b26-60a134d28724 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:14.308234 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e766e-644f-4c16-9b26-60a134d28724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:14.308515 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e766e-644f-4c16-9b26-60a134d28724 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:14.309223 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e766e-644f-4c16-9b26-60a134d28724 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:14.311186 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e766e-644f-4c16-9b26-60a134d28724 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:14.419259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1412/2817] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:26:14 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 15:26:14.423564 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90c3c0-903c-4a96-9129-4de9faca2861 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:14.434320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90c3c0-903c-4a96-9129-4de9faca2861 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:14.493957 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e431fda-51f4-4301-ba24-78f52bae6258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:14.494672 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e431fda-51f4-4301-ba24-78f52bae6258 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:14.494962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e431fda-51f4-4301-ba24-78f52bae6258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:14.495266 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e431fda-51f4-4301-ba24-78f52bae6258 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:14.495960 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e431fda-51f4-4301-ba24-78f52bae6258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:14.498061 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e431fda-51f4-4301-ba24-78f52bae6258 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:14.577848 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1406/2818] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:26:14 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:26:14.582451 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e766e-644f-4c16-9b26-60a134d28724 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:14.593922 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e766e-644f-4c16-9b26-60a134d28724 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:14.598903 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2fbf99-075e-40b7-b524-2d4e952db51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:14.599567 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fbf99-075e-40b7-b524-2d4e952db51d admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:14.599853 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fbf99-075e-40b7-b524-2d4e952db51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:14.600146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fbf99-075e-40b7-b524-2d4e952db51d admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:14.600879 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2fbf99-075e-40b7-b524-2d4e952db51d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:14.603173 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2fbf99-075e-40b7-b524-2d4e952db51d admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:14.614925 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1413/2819] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:26:14.619551 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e431fda-51f4-4301-ba24-78f52bae6258 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:14.629944 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e431fda-51f4-4301-ba24-78f52bae6258 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:14.634941 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de84a7-c250-4386-8157-3598ff59519e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:14.635603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de84a7-c250-4386-8157-3598ff59519e admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:14.635890 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de84a7-c250-4386-8157-3598ff59519e admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:14.636171 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de84a7-c250-4386-8157-3598ff59519e admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:14.636896 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de84a7-c250-4386-8157-3598ff59519e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:14.638591 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de84a7-c250-4386-8157-3598ff59519e admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:14.639981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:14.650478 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1407/2820] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:26:14.654919 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2fbf99-075e-40b7-b524-2d4e952db51d admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:14.665506 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2fbf99-075e-40b7-b524-2d4e952db51d admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:14.670267 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b1c42-3ec9-46f3-9238-273f1012971a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:14.671118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b1c42-3ec9-46f3-9238-273f1012971a admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:14.671401 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b1c42-3ec9-46f3-9238-273f1012971a admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:14.671709 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b1c42-3ec9-46f3-9238-273f1012971a admin admin] PATH_INFO: `/v3/projects/56ab052d89874f499b2020d23539bfb2/users/90ba497b61cc4ce9aeaef99da9aec6af/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:14.686482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b1c42-3ec9-46f3-9238-273f1012971a admin admin] RBAC: Authorizing `identity:create_grant(project_id=56ab052d89874f499b2020d23539bfb2, user_id=90ba497b61cc4ce9aeaef99da9aec6af, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:14.689178 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b1c42-3ec9-46f3-9238-273f1012971a admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:14.774886 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1414/2821] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:26:14 2022] PUT /identity/v3/projects/56ab052d89874f499b2020d23539bfb2/users/90ba497b61cc4ce9aeaef99da9aec6af/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:14.787134 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cb97b-807f-4902-b566-0f27c48a7e17 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:14.787425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cb97b-807f-4902-b566-0f27c48a7e17 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:14.787707 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cb97b-807f-4902-b566-0f27c48a7e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:14.816405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-554cb97b-807f-4902-b566-0f27c48a7e17 None None] MFA Rules not processed for user `90ba497b61cc4ce9aeaef99da9aec6af`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:14.817643 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-554cb97b-807f-4902-b566-0f27c48a7e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:14.882549 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1408/2822] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:26:14.888013 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b1c42-3ec9-46f3-9238-273f1012971a admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:14.929907 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:14.954262 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b1c42-3ec9-46f3-9238-273f1012971a admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:14.959415 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b66576-77cc-4f70-889b-a54a40dd7006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:14.960311 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b66576-77cc-4f70-889b-a54a40dd7006 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:14.961027 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b66576-77cc-4f70-889b-a54a40dd7006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:14.961961 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b66576-77cc-4f70-889b-a54a40dd7006 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:14.962797 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b66576-77cc-4f70-889b-a54a40dd7006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:14.964717 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b66576-77cc-4f70-889b-a54a40dd7006 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:15.062605 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1415/2823] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:26:14 2022] POST /identity/v3/projects => generated 365 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:26:15.067004 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554cb97b-807f-4902-b566-0f27c48a7e17 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:15.078448 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554cb97b-807f-4902-b566-0f27c48a7e17 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:15.136280 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a43ee-f26f-4386-bba6-46abbca8ad60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:15.137159 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a43ee-f26f-4386-bba6-46abbca8ad60 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:15.137276 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a43ee-f26f-4386-bba6-46abbca8ad60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:15.137906 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a43ee-f26f-4386-bba6-46abbca8ad60 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:15.138186 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a43ee-f26f-4386-bba6-46abbca8ad60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:15.140328 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a43ee-f26f-4386-bba6-46abbca8ad60 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:15.237447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1409/2824] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:26:15 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:26:15.241964 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b66576-77cc-4f70-889b-a54a40dd7006 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:15.252770 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b66576-77cc-4f70-889b-a54a40dd7006 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:15.257626 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7221d-44a3-4d32-b447-7db7adea76e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:15.258388 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7221d-44a3-4d32-b447-7db7adea76e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:15.258734 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7221d-44a3-4d32-b447-7db7adea76e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:15.259005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7221d-44a3-4d32-b447-7db7adea76e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:15.259736 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7221d-44a3-4d32-b447-7db7adea76e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:15.261357 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7221d-44a3-4d32-b447-7db7adea76e7 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:15.272566 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1416/2825] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:26:15.277678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a43ee-f26f-4386-bba6-46abbca8ad60 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:15.287981 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a43ee-f26f-4386-bba6-46abbca8ad60 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:15.293243 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56893bdf-3f54-408d-9d85-45a00c6e6bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:15.294127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56893bdf-3f54-408d-9d85-45a00c6e6bf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:15.294415 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56893bdf-3f54-408d-9d85-45a00c6e6bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:15.294738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56893bdf-3f54-408d-9d85-45a00c6e6bf4 admin admin] PATH_INFO: `/v3/projects/c0a3ce48b73f4c968aea2f7d442c2ee6/users/7dca2d492c1a476f863e4ff4625194c5/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:15.310309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56893bdf-3f54-408d-9d85-45a00c6e6bf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0a3ce48b73f4c968aea2f7d442c2ee6, user_id=7dca2d492c1a476f863e4ff4625194c5, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:15.315406 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56893bdf-3f54-408d-9d85-45a00c6e6bf4 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:15.403797 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1410/2826] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:26:15 2022] PUT /identity/v3/projects/c0a3ce48b73f4c968aea2f7d442c2ee6/users/7dca2d492c1a476f863e4ff4625194c5/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:15.407933 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7221d-44a3-4d32-b447-7db7adea76e7 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:15.418470 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7221d-44a3-4d32-b447-7db7adea76e7 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:15.475182 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd04af4-55b0-4b2d-b7b5-a462693d84b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:15.475855 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd04af4-55b0-4b2d-b7b5-a462693d84b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:15.476147 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd04af4-55b0-4b2d-b7b5-a462693d84b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:15.476436 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd04af4-55b0-4b2d-b7b5-a462693d84b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:15.477874 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd04af4-55b0-4b2d-b7b5-a462693d84b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:15.479519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd04af4-55b0-4b2d-b7b5-a462693d84b6 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:15.489573 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1417/2827] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:26:15.493885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56893bdf-3f54-408d-9d85-45a00c6e6bf4 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:15.504874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56893bdf-3f54-408d-9d85-45a00c6e6bf4 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:15.509776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5f4a6-e0d8-4cba-bfe0-37dfb29a2703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:15.510727 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5f4a6-e0d8-4cba-bfe0-37dfb29a2703 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:15.511015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5f4a6-e0d8-4cba-bfe0-37dfb29a2703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:15.511295 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5f4a6-e0d8-4cba-bfe0-37dfb29a2703 admin admin] PATH_INFO: `/v3/projects/c0a3ce48b73f4c968aea2f7d442c2ee6/users/7dca2d492c1a476f863e4ff4625194c5/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:15.513320 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f5f4a6-e0d8-4cba-bfe0-37dfb29a2703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0a3ce48b73f4c968aea2f7d442c2ee6, user_id=7dca2d492c1a476f863e4ff4625194c5, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:15.516030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f5f4a6-e0d8-4cba-bfe0-37dfb29a2703 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:15.527219 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1411/2828] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:26:15 2022] PUT /identity/v3/projects/c0a3ce48b73f4c968aea2f7d442c2ee6/users/7dca2d492c1a476f863e4ff4625194c5/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:15.537737 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767c139-7572-4e5c-92d6-ef18a3300639 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:15.538032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767c139-7572-4e5c-92d6-ef18a3300639 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:15.538344 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767c139-7572-4e5c-92d6-ef18a3300639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:15.566616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-3767c139-7572-4e5c-92d6-ef18a3300639 None None] MFA Rules not processed for user `7dca2d492c1a476f863e4ff4625194c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:15.567843 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-3767c139-7572-4e5c-92d6-ef18a3300639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:15.640739 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1418/2829] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:15.653647 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5f4a6-e0d8-4cba-bfe0-37dfb29a2703 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:15.655581 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-13f5f4a6-e0d8-4cba-bfe0-37dfb29a2703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:15.699678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:15.715615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5f4a6-e0d8-4cba-bfe0-37dfb29a2703 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:15.719341 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3d9cd-74b4-454c-b928-b2a39cf7fd5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:15.719988 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3d9cd-74b4-454c-b928-b2a39cf7fd5e service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:15.720272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3d9cd-74b4-454c-b928-b2a39cf7fd5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:15.720553 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3d9cd-74b4-454c-b928-b2a39cf7fd5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:15.730080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3d9cd-74b4-454c-b928-b2a39cf7fd5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:15.732089 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3d9cd-74b4-454c-b928-b2a39cf7fd5e service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:15.746625 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1412/2830] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:26:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:26:15.951890 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767c139-7572-4e5c-92d6-ef18a3300639 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:15.962835 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767c139-7572-4e5c-92d6-ef18a3300639 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:15.966416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a912bd-9a21-4a78-9fd8-d0fdd26923b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:15.967092 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a912bd-9a21-4a78-9fd8-d0fdd26923b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:15.967377 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a912bd-9a21-4a78-9fd8-d0fdd26923b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:15.967656 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a912bd-9a21-4a78-9fd8-d0fdd26923b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:15.999623 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:16.028141 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a912bd-9a21-4a78-9fd8-d0fdd26923b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:16.029683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a912bd-9a21-4a78-9fd8-d0fdd26923b8 service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:16.039285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1419/2831] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:26:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:26:17.947667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3d9cd-74b4-454c-b928-b2a39cf7fd5e service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:17.949126 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-61c3d9cd-74b4-454c-b928-b2a39cf7fd5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:18.022841 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3d9cd-74b4-454c-b928-b2a39cf7fd5e service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:18.028147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94eb359-be73-4081-b802-93b473a65f30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76cc0ae48b5a49c6a7a1c11a08c389e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:18.028793 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94eb359-be73-4081-b802-93b473a65f30 service placement] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:18.029093 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94eb359-be73-4081-b802-93b473a65f30 service placement] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:18.029354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94eb359-be73-4081-b802-93b473a65f30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:18.031151 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-b94eb359-be73-4081-b802-93b473a65f30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:18.043437 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94eb359-be73-4081-b802-93b473a65f30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:18.045016 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94eb359-be73-4081-b802-93b473a65f30 service placement] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:18.056425 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1413/2832] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:26:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:26:19.639708 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a912bd-9a21-4a78-9fd8-d0fdd26923b8 service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:19.702455 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a912bd-9a21-4a78-9fd8-d0fdd26923b8 service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:19.707103 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296a991-8528-49e2-873c-e027bbacb0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:19.708002 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296a991-8528-49e2-873c-e027bbacb0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:19.708287 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296a991-8528-49e2-873c-e027bbacb0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:19.708565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296a991-8528-49e2-873c-e027bbacb0b0 admin admin] PATH_INFO: `/v3/users/90ba497b61cc4ce9aeaef99da9aec6af` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:19.709805 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296a991-8528-49e2-873c-e027bbacb0b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90ba497b61cc4ce9aeaef99da9aec6af)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:19.712800 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296a991-8528-49e2-873c-e027bbacb0b0 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:19.952460 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b296a991-8528-49e2-873c-e027bbacb0b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90ba497b61cc4ce9aeaef99da9aec6af'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:19.953128 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b296a991-8528-49e2-873c-e027bbacb0b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90ba497b61cc4ce9aeaef99da9aec6af'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:19.953748 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b296a991-8528-49e2-873c-e027bbacb0b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90ba497b61cc4ce9aeaef99da9aec6af'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:20.009590 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b296a991-8528-49e2-873c-e027bbacb0b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90ba497b61cc4ce9aeaef99da9aec6af'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:20.022613 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-b296a991-8528-49e2-873c-e027bbacb0b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90ba497b61cc4ce9aeaef99da9aec6af'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:20.035309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1420/2833] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:26:19 2022] DELETE /identity/v3/users/90ba497b61cc4ce9aeaef99da9aec6af => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:26:20.040080 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e4925-282c-4eb7-a901-e3701618cb4b None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:20.040620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e4925-282c-4eb7-a901-e3701618cb4b None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:20.040751 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e4925-282c-4eb7-a901-e3701618cb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:20.058311 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-2f3e4925-282c-4eb7-a901-e3701618cb4b None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:20.059575 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-2f3e4925-282c-4eb7-a901-e3701618cb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:20.119924 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1414/2834] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:20.137969 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296a991-8528-49e2-873c-e027bbacb0b0 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:20.139472 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b296a991-8528-49e2-873c-e027bbacb0b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:20.199908 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296a991-8528-49e2-873c-e027bbacb0b0 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:20.203576 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd7db6-c061-4ca1-a1bc-453d500c6804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:20.204267 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd7db6-c061-4ca1-a1bc-453d500c6804 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:20.204555 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd7db6-c061-4ca1-a1bc-453d500c6804 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:20.204912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd7db6-c061-4ca1-a1bc-453d500c6804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:20.212179 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:20.216138 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd7db6-c061-4ca1-a1bc-453d500c6804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:20.217601 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd7db6-c061-4ca1-a1bc-453d500c6804 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:20.228943 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1421/2835] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:26:20.717096 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e4925-282c-4eb7-a901-e3701618cb4b None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:20.722725 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-2f3e4925-282c-4eb7-a901-e3701618cb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:20.740447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e4925-282c-4eb7-a901-e3701618cb4b None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:20.747845 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0b17b-70af-4af8-8c07-490e14ab1119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:20.748723 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0b17b-70af-4af8-8c07-490e14ab1119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:20.749054 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0b17b-70af-4af8-8c07-490e14ab1119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:20.749355 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0b17b-70af-4af8-8c07-490e14ab1119 admin admin] PATH_INFO: `/v3/projects/56ab052d89874f499b2020d23539bfb2` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:20.750483 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0b17b-70af-4af8-8c07-490e14ab1119 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56ab052d89874f499b2020d23539bfb2)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:20.752571 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0b17b-70af-4af8-8c07-490e14ab1119 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:20.890144 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-58b0b17b-70af-4af8-8c07-490e14ab1119 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56ab052d89874f499b2020d23539bfb2', 'request_id': 'req-58b0b17b-70af-4af8-8c07-490e14ab1119'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:20.896528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-58b0b17b-70af-4af8-8c07-490e14ab1119 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56ab052d89874f499b2020d23539bfb2', 'request_id': 'req-58b0b17b-70af-4af8-8c07-490e14ab1119'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:20.971576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-58b0b17b-70af-4af8-8c07-490e14ab1119 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56ab052d89874f499b2020d23539bfb2', 'request_id': 'req-58b0b17b-70af-4af8-8c07-490e14ab1119'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:20.976007 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:20.983683 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-58b0b17b-70af-4af8-8c07-490e14ab1119 admin admin] The token cache is being invalidate because project 56ab052d89874f499b2020d23539bfb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:26:20.984000 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-58b0b17b-70af-4af8-8c07-490e14ab1119 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:20.984619 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-58b0b17b-70af-4af8-8c07-490e14ab1119 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:20.986253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1415/2836] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:26:20 2022] DELETE /identity/v3/projects/56ab052d89874f499b2020d23539bfb2 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:20.990616 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd7db6-c061-4ca1-a1bc-453d500c6804 service neutron] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:21.001312 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd7db6-c061-4ca1-a1bc-453d500c6804 service neutron] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:21.065459 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ed328-5c4f-4666-be4c-197ae46549d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:21.066342 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ed328-5c4f-4666-be4c-197ae46549d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:21.066660 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ed328-5c4f-4666-be4c-197ae46549d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:21.066941 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ed328-5c4f-4666-be4c-197ae46549d5 admin admin] PATH_INFO: `/v3/users/7dca2d492c1a476f863e4ff4625194c5` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:21.068259 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ed328-5c4f-4666-be4c-197ae46549d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dca2d492c1a476f863e4ff4625194c5)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:21.070268 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ed328-5c4f-4666-be4c-197ae46549d5 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:21.264740 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-804ed328-5c4f-4666-be4c-197ae46549d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dca2d492c1a476f863e4ff4625194c5'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:21.265498 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-804ed328-5c4f-4666-be4c-197ae46549d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dca2d492c1a476f863e4ff4625194c5'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:21.266146 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-804ed328-5c4f-4666-be4c-197ae46549d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dca2d492c1a476f863e4ff4625194c5'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:21.334305 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-804ed328-5c4f-4666-be4c-197ae46549d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dca2d492c1a476f863e4ff4625194c5'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:21.348335 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-804ed328-5c4f-4666-be4c-197ae46549d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dca2d492c1a476f863e4ff4625194c5'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:21.362113 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1422/2837] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:26:20 2022] DELETE /identity/v3/users/7dca2d492c1a476f863e4ff4625194c5 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:21.811007 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0b17b-70af-4af8-8c07-490e14ab1119 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:21.813220 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-58b0b17b-70af-4af8-8c07-490e14ab1119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:21.831898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0b17b-70af-4af8-8c07-490e14ab1119 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:21.911828 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9f223-445d-49e7-864b-155b24eec286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:21.912813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9f223-445d-49e7-864b-155b24eec286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:21.913097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9f223-445d-49e7-864b-155b24eec286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:21.913375 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9f223-445d-49e7-864b-155b24eec286 admin admin] PATH_INFO: `/v3/projects/c0a3ce48b73f4c968aea2f7d442c2ee6` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:21.914620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9f223-445d-49e7-864b-155b24eec286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0a3ce48b73f4c968aea2f7d442c2ee6)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:21.916528 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9f223-445d-49e7-864b-155b24eec286 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:22.015284 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-89f9f223-445d-49e7-864b-155b24eec286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0a3ce48b73f4c968aea2f7d442c2ee6', 'request_id': 'req-89f9f223-445d-49e7-864b-155b24eec286'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:22.023741 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-89f9f223-445d-49e7-864b-155b24eec286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0a3ce48b73f4c968aea2f7d442c2ee6', 'request_id': 'req-89f9f223-445d-49e7-864b-155b24eec286'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:22.085445 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-89f9f223-445d-49e7-864b-155b24eec286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0a3ce48b73f4c968aea2f7d442c2ee6', 'request_id': 'req-89f9f223-445d-49e7-864b-155b24eec286'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:22.096941 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-89f9f223-445d-49e7-864b-155b24eec286 admin admin] The token cache is being invalidate because project c0a3ce48b73f4c968aea2f7d442c2ee6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:26:22.097230 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-89f9f223-445d-49e7-864b-155b24eec286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:22.097872 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-89f9f223-445d-49e7-864b-155b24eec286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:22.099466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1416/2838] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:26:21 2022] DELETE /identity/v3/projects/c0a3ce48b73f4c968aea2f7d442c2ee6 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:22.104933 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac27ae-ca11-4e03-a3b7-70320feeeb9c None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:22.105452 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac27ae-ca11-4e03-a3b7-70320feeeb9c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:22.105587 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac27ae-ca11-4e03-a3b7-70320feeeb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:22.123780 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-ecac27ae-ca11-4e03-a3b7-70320feeeb9c None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:22.125024 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-ecac27ae-ca11-4e03-a3b7-70320feeeb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:22.186107 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1423/2839] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:22.194434 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426ca85-2b70-4a59-abdb-c751f7cd4687 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:22.194771 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426ca85-2b70-4a59-abdb-c751f7cd4687 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:22.195801 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426ca85-2b70-4a59-abdb-c751f7cd4687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:22.214210 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-1426ca85-2b70-4a59-abdb-c751f7cd4687 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:22.215608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-1426ca85-2b70-4a59-abdb-c751f7cd4687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:22.224026 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1417/2840] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:22.228756 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecac27ae-ca11-4e03-a3b7-70320feeeb9c None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:22.238423 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecac27ae-ca11-4e03-a3b7-70320feeeb9c None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:22.243871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507bb08-56b2-4e43-b7d9-dc8cd4b67716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:22.244528 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507bb08-56b2-4e43-b7d9-dc8cd4b67716 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:22.252881 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507bb08-56b2-4e43-b7d9-dc8cd4b67716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:22.252881 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507bb08-56b2-4e43-b7d9-dc8cd4b67716 admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:22.252881 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507bb08-56b2-4e43-b7d9-dc8cd4b67716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:26:22.252881 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507bb08-56b2-4e43-b7d9-dc8cd4b67716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:22.252881 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507bb08-56b2-4e43-b7d9-dc8cd4b67716 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:22.256586 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:22.260661 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1424/2841] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:26:22.265193 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426ca85-2b70-4a59-abdb-c751f7cd4687 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:22.275920 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426ca85-2b70-4a59-abdb-c751f7cd4687 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:22.280714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b22c8-52b7-4861-8542-28656a348c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:22.281374 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b22c8-52b7-4861-8542-28656a348c64 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:22.281654 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b22c8-52b7-4861-8542-28656a348c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:22.281927 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b22c8-52b7-4861-8542-28656a348c64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:22.282646 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b22c8-52b7-4861-8542-28656a348c64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:22.284644 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b22c8-52b7-4861-8542-28656a348c64 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:22.392837 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1418/2842] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:26:22 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:26:22.397275 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507bb08-56b2-4e43-b7d9-dc8cd4b67716 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:22.408000 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507bb08-56b2-4e43-b7d9-dc8cd4b67716 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:22.466469 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183651e0-9106-4897-8eeb-cab870e2164b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:22.467206 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183651e0-9106-4897-8eeb-cab870e2164b admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:22.467543 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183651e0-9106-4897-8eeb-cab870e2164b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:22.467823 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183651e0-9106-4897-8eeb-cab870e2164b admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:22.468510 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183651e0-9106-4897-8eeb-cab870e2164b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:22.470430 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183651e0-9106-4897-8eeb-cab870e2164b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:22.585466 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1425/2843] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:26:22 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:26:22.589946 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b22c8-52b7-4861-8542-28656a348c64 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:22.600603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b22c8-52b7-4861-8542-28656a348c64 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:22.606272 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ab7e0-c53d-407a-8602-0dff7d13b4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:22.607005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ab7e0-c53d-407a-8602-0dff7d13b4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:22.607287 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ab7e0-c53d-407a-8602-0dff7d13b4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:22.607576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ab7e0-c53d-407a-8602-0dff7d13b4d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:22.608280 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ab7e0-c53d-407a-8602-0dff7d13b4d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:22.609886 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ab7e0-c53d-407a-8602-0dff7d13b4d7 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:22.620987 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1419/2844] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 15:26:22.625263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183651e0-9106-4897-8eeb-cab870e2164b admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:22.636091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183651e0-9106-4897-8eeb-cab870e2164b admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:22.641122 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806b6e5-00af-4a9c-bb3c-daea3f7d2eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:22.641944 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806b6e5-00af-4a9c-bb3c-daea3f7d2eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:22.642230 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806b6e5-00af-4a9c-bb3c-daea3f7d2eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:22.642514 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806b6e5-00af-4a9c-bb3c-daea3f7d2eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:22.643352 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806b6e5-00af-4a9c-bb3c-daea3f7d2eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:22.644906 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806b6e5-00af-4a9c-bb3c-daea3f7d2eb2 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:22.656277 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1426/2845] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:26:22.660593 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ab7e0-c53d-407a-8602-0dff7d13b4d7 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:22.671328 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ab7e0-c53d-407a-8602-0dff7d13b4d7 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:22.676127 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f457a-aeae-4645-8596-12f11589def5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:22.677005 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f457a-aeae-4645-8596-12f11589def5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:22.677291 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f457a-aeae-4645-8596-12f11589def5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:22.677567 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f457a-aeae-4645-8596-12f11589def5 admin admin] PATH_INFO: `/v3/projects/c3614c39118c493e9e4efac872184738/users/41218472d75c42eb8480b95ef44fc6f7/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:22.692950 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f457a-aeae-4645-8596-12f11589def5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3614c39118c493e9e4efac872184738, user_id=41218472d75c42eb8480b95ef44fc6f7, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:22.695722 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f457a-aeae-4645-8596-12f11589def5 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:22.791960 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1420/2846] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:26:22 2022] PUT /identity/v3/projects/c3614c39118c493e9e4efac872184738/users/41218472d75c42eb8480b95ef44fc6f7/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:22.803754 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e201b75-710f-4b28-872f-b1081a5071e2 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:22.804052 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e201b75-710f-4b28-872f-b1081a5071e2 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:22.804979 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e201b75-710f-4b28-872f-b1081a5071e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:22.831439 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-6e201b75-710f-4b28-872f-b1081a5071e2 None None] MFA Rules not processed for user `41218472d75c42eb8480b95ef44fc6f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:22.832675 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-6e201b75-710f-4b28-872f-b1081a5071e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:22.899633 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1427/2847] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:22.912157 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f457a-aeae-4645-8596-12f11589def5 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:22.914671 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a16f457a-aeae-4645-8596-12f11589def5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:22.938278 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:22.978826 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f457a-aeae-4645-8596-12f11589def5 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:22.983576 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a6e0b-77dd-44ce-bba9-c0e7ff435ebb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:22.984284 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a6e0b-77dd-44ce-bba9-c0e7ff435ebb service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:22.984568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a6e0b-77dd-44ce-bba9-c0e7ff435ebb service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:22.984869 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a6e0b-77dd-44ce-bba9-c0e7ff435ebb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:22.996128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a6e0b-77dd-44ce-bba9-c0e7ff435ebb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:22.997604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a6e0b-77dd-44ce-bba9-c0e7ff435ebb service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:23.007064 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1421/2848] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:26:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:26:32.895867 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:33.045354 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=17 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:33.057412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e201b75-710f-4b28-872f-b1081a5071e2 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:33.121482 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e201b75-710f-4b28-872f-b1081a5071e2 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:33.126111 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01735cc-3438-41fd-9b30-17efb0863785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:33.127109 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01735cc-3438-41fd-9b30-17efb0863785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:33.127346 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01735cc-3438-41fd-9b30-17efb0863785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:33.127582 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01735cc-3438-41fd-9b30-17efb0863785 admin admin] PATH_INFO: `/v3/users/41218472d75c42eb8480b95ef44fc6f7` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:33.128792 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01735cc-3438-41fd-9b30-17efb0863785 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41218472d75c42eb8480b95ef44fc6f7)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:33.130746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01735cc-3438-41fd-9b30-17efb0863785 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:33.305215 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-e01735cc-3438-41fd-9b30-17efb0863785 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41218472d75c42eb8480b95ef44fc6f7'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:33.305745 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-e01735cc-3438-41fd-9b30-17efb0863785 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41218472d75c42eb8480b95ef44fc6f7'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:33.306396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-e01735cc-3438-41fd-9b30-17efb0863785 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41218472d75c42eb8480b95ef44fc6f7'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:33.387830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-e01735cc-3438-41fd-9b30-17efb0863785 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41218472d75c42eb8480b95ef44fc6f7'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:33.402629 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-e01735cc-3438-41fd-9b30-17efb0863785 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41218472d75c42eb8480b95ef44fc6f7'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:33.416711 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1428/2849] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:26:33 2022] DELETE /identity/v3/users/41218472d75c42eb8480b95ef44fc6f7 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:33.421462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30b464-0670-4957-bf49-f449b4be99d8 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:33.421756 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30b464-0670-4957-bf49-f449b4be99d8 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:33.422039 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30b464-0670-4957-bf49-f449b4be99d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:33.439748 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-9c30b464-0670-4957-bf49-f449b4be99d8 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:33.440985 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-9c30b464-0670-4957-bf49-f449b4be99d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:33.500798 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1422/2850] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:33.510968 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01735cc-3438-41fd-9b30-17efb0863785 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:33.512517 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-e01735cc-3438-41fd-9b30-17efb0863785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:33.570758 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01735cc-3438-41fd-9b30-17efb0863785 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:33.574743 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12142459-acd8-4d53-938d-a97349db6dd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:33.575392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12142459-acd8-4d53-938d-a97349db6dd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:33.575678 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12142459-acd8-4d53-938d-a97349db6dd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:33.575958 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12142459-acd8-4d53-938d-a97349db6dd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:33.586125 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12142459-acd8-4d53-938d-a97349db6dd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:33.587612 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12142459-acd8-4d53-938d-a97349db6dd2 service neutron] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:33.597465 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1429/2851] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:26:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 15:26:34.058897 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30b464-0670-4957-bf49-f449b4be99d8 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:34.060473 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-9c30b464-0670-4957-bf49-f449b4be99d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:34.078068 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30b464-0670-4957-bf49-f449b4be99d8 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:34.082343 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1deda4d-ad8f-4203-9abf-b1a90d035a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:34.083336 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1deda4d-ad8f-4203-9abf-b1a90d035a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:34.083778 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1deda4d-ad8f-4203-9abf-b1a90d035a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:34.084153 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1deda4d-ad8f-4203-9abf-b1a90d035a0e admin admin] PATH_INFO: `/v3/projects/c3614c39118c493e9e4efac872184738` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:34.086030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1deda4d-ad8f-4203-9abf-b1a90d035a0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3614c39118c493e9e4efac872184738)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:34.088360 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1deda4d-ad8f-4203-9abf-b1a90d035a0e admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:34.241520 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b1deda4d-ad8f-4203-9abf-b1a90d035a0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3614c39118c493e9e4efac872184738', 'request_id': 'req-b1deda4d-ad8f-4203-9abf-b1a90d035a0e'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:34.247846 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b1deda4d-ad8f-4203-9abf-b1a90d035a0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3614c39118c493e9e4efac872184738', 'request_id': 'req-b1deda4d-ad8f-4203-9abf-b1a90d035a0e'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:34.305304 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b1deda4d-ad8f-4203-9abf-b1a90d035a0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3614c39118c493e9e4efac872184738', 'request_id': 'req-b1deda4d-ad8f-4203-9abf-b1a90d035a0e'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:34.316666 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b1deda4d-ad8f-4203-9abf-b1a90d035a0e admin admin] The token cache is being invalidate because project c3614c39118c493e9e4efac872184738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80303) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:26:34.316959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b1deda4d-ad8f-4203-9abf-b1a90d035a0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:34.317615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-b1deda4d-ad8f-4203-9abf-b1a90d035a0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:34.319436 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1423/2852] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:26:34 2022] DELETE /identity/v3/projects/c3614c39118c493e9e4efac872184738 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:26:34.324659 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb9106-5590-49de-8f93-f65f7f5fe241 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:34.325237 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb9106-5590-49de-8f93-f65f7f5fe241 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:34.325237 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb9106-5590-49de-8f93-f65f7f5fe241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:34.343710 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-5ebb9106-5590-49de-8f93-f65f7f5fe241 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:34.344861 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-5ebb9106-5590-49de-8f93-f65f7f5fe241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:34.362714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:34.404912 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1430/2853] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:34.412839 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb94997-718e-4be7-bfd8-1a4c8a9905d3 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:34.413700 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb94997-718e-4be7-bfd8-1a4c8a9905d3 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:34.414466 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb94997-718e-4be7-bfd8-1a4c8a9905d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:34.433901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-0bb94997-718e-4be7-bfd8-1a4c8a9905d3 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:34.435224 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-0bb94997-718e-4be7-bfd8-1a4c8a9905d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:34.443608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1424/2854] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:34.448973 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb9106-5590-49de-8f93-f65f7f5fe241 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:34.458589 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb9106-5590-49de-8f93-f65f7f5fe241 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:34.464309 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e77e6-3f2a-45a3-9b96-57be88ce47ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:34.465007 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e77e6-3f2a-45a3-9b96-57be88ce47ce admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:34.465302 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e77e6-3f2a-45a3-9b96-57be88ce47ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:34.465606 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e77e6-3f2a-45a3-9b96-57be88ce47ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:34.466368 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e77e6-3f2a-45a3-9b96-57be88ce47ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80304) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:26:34.466716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e77e6-3f2a-45a3-9b96-57be88ce47ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:34.468370 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e77e6-3f2a-45a3-9b96-57be88ce47ce admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:34.480862 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1431/2855] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:26:34.485283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb94997-718e-4be7-bfd8-1a4c8a9905d3 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:34.494976 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb94997-718e-4be7-bfd8-1a4c8a9905d3 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:34.499712 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547599d4-29e0-4ba9-b1b4-2be35abc6419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:34.500372 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547599d4-29e0-4ba9-b1b4-2be35abc6419 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:34.500660 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547599d4-29e0-4ba9-b1b4-2be35abc6419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:34.500937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547599d4-29e0-4ba9-b1b4-2be35abc6419 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:34.501634 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547599d4-29e0-4ba9-b1b4-2be35abc6419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:34.503591 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547599d4-29e0-4ba9-b1b4-2be35abc6419 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:34.587984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1425/2856] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:26:34 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:26:34.595069 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e77e6-3f2a-45a3-9b96-57be88ce47ce admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:34.608646 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e77e6-3f2a-45a3-9b96-57be88ce47ce admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:34.665925 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bcebb-c06c-4fe4-8b2d-905701b279a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:34.666617 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bcebb-c06c-4fe4-8b2d-905701b279a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:34.666905 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bcebb-c06c-4fe4-8b2d-905701b279a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:34.667184 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bcebb-c06c-4fe4-8b2d-905701b279a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:34.667870 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bcebb-c06c-4fe4-8b2d-905701b279a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:34.669793 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bcebb-c06c-4fe4-8b2d-905701b279a4 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:34.739825 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1432/2857] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:26:34 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:26:34.744618 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547599d4-29e0-4ba9-b1b4-2be35abc6419 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:34.755190 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547599d4-29e0-4ba9-b1b4-2be35abc6419 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:34.760395 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f716e6e-1402-4689-afcd-56cec32ba7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:34.761058 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f716e6e-1402-4689-afcd-56cec32ba7a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:34.761347 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f716e6e-1402-4689-afcd-56cec32ba7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:34.761676 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f716e6e-1402-4689-afcd-56cec32ba7a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:34.762404 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f716e6e-1402-4689-afcd-56cec32ba7a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:34.764071 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f716e6e-1402-4689-afcd-56cec32ba7a1 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:34.775531 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1426/2858] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:26:34.779571 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bcebb-c06c-4fe4-8b2d-905701b279a4 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:34.790248 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bcebb-c06c-4fe4-8b2d-905701b279a4 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:34.795019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5747d-98e4-40d2-86a4-cc13a128dff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:34.795682 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5747d-98e4-40d2-86a4-cc13a128dff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:34.795966 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5747d-98e4-40d2-86a4-cc13a128dff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:34.796271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5747d-98e4-40d2-86a4-cc13a128dff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:34.796994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5747d-98e4-40d2-86a4-cc13a128dff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:34.798612 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5747d-98e4-40d2-86a4-cc13a128dff8 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:34.809382 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1433/2859] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:26:34.813781 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f716e6e-1402-4689-afcd-56cec32ba7a1 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:34.824548 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f716e6e-1402-4689-afcd-56cec32ba7a1 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:34.829374 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da16449-ef40-4e27-a84a-27de5d288904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:34.830293 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da16449-ef40-4e27-a84a-27de5d288904 admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:34.830608 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da16449-ef40-4e27-a84a-27de5d288904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:34.830884 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da16449-ef40-4e27-a84a-27de5d288904 admin admin] PATH_INFO: `/v3/projects/20432d0a46bc46129034faec8269572a/users/fcb82d098513497eb27c57fb1fec6c51/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:34.846447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da16449-ef40-4e27-a84a-27de5d288904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20432d0a46bc46129034faec8269572a, user_id=fcb82d098513497eb27c57fb1fec6c51, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:34.849047 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da16449-ef40-4e27-a84a-27de5d288904 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:34.922889 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1427/2860] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:26:34 2022] PUT /identity/v3/projects/20432d0a46bc46129034faec8269572a/users/fcb82d098513497eb27c57fb1fec6c51/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:34.936140 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ec533-0edf-4a97-af01-65af3053a81c None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:34.936620 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ec533-0edf-4a97-af01-65af3053a81c None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:34.936734 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ec533-0edf-4a97-af01-65af3053a81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:34.964907 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-2b4ec533-0edf-4a97-af01-65af3053a81c None None] MFA Rules not processed for user `fcb82d098513497eb27c57fb1fec6c51`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:34.966149 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-2b4ec533-0edf-4a97-af01-65af3053a81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:35.033565 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1434/2861] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:35.038855 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da16449-ef40-4e27-a84a-27de5d288904 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:35.060002 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:35.103363 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da16449-ef40-4e27-a84a-27de5d288904 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:35.108097 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a9c62-ebda-40df-9bb1-b48d8bac8bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:35.114074 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a9c62-ebda-40df-9bb1-b48d8bac8bbc admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:35.114074 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a9c62-ebda-40df-9bb1-b48d8bac8bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:35.114074 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a9c62-ebda-40df-9bb1-b48d8bac8bbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:35.114074 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a9c62-ebda-40df-9bb1-b48d8bac8bbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:35.114074 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a9c62-ebda-40df-9bb1-b48d8bac8bbc admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:35.198931 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1428/2862] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:26:35 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:26:35.203184 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ec533-0edf-4a97-af01-65af3053a81c None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:35.214681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ec533-0edf-4a97-af01-65af3053a81c None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:35.228046 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:35.272804 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c6f80-1554-4bfb-a94c-f9799f281282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:35.273579 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c6f80-1554-4bfb-a94c-f9799f281282 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:35.273869 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c6f80-1554-4bfb-a94c-f9799f281282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:35.274157 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c6f80-1554-4bfb-a94c-f9799f281282 admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:35.274897 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c6f80-1554-4bfb-a94c-f9799f281282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:35.276846 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c6f80-1554-4bfb-a94c-f9799f281282 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:35.386005 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1435/2863] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:26:35 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:26:35.390112 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a9c62-ebda-40df-9bb1-b48d8bac8bbc admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:35.400776 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a9c62-ebda-40df-9bb1-b48d8bac8bbc admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:35.405380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a682a1-04f8-4946-8a79-b27e0ce375ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:35.406033 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a682a1-04f8-4946-8a79-b27e0ce375ab admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:35.406338 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a682a1-04f8-4946-8a79-b27e0ce375ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:35.406650 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a682a1-04f8-4946-8a79-b27e0ce375ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:35.411721 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a682a1-04f8-4946-8a79-b27e0ce375ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:35.411721 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a682a1-04f8-4946-8a79-b27e0ce375ab admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:35.420049 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1429/2864] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:26:35.424126 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c6f80-1554-4bfb-a94c-f9799f281282 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:35.435310 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c6f80-1554-4bfb-a94c-f9799f281282 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:35.440101 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a42bd-9bdc-44e9-85d6-58b14f3f0a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:35.440756 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a42bd-9bdc-44e9-85d6-58b14f3f0a5b admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:35.441040 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a42bd-9bdc-44e9-85d6-58b14f3f0a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:35.441317 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a42bd-9bdc-44e9-85d6-58b14f3f0a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:35.442044 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a42bd-9bdc-44e9-85d6-58b14f3f0a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:35.443765 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a42bd-9bdc-44e9-85d6-58b14f3f0a5b admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:35.454072 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1436/2865] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:26:35.458468 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a682a1-04f8-4946-8a79-b27e0ce375ab admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:35.468739 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a682a1-04f8-4946-8a79-b27e0ce375ab admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:35.481693 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccc0b3-2379-47c7-a6a5-1bcdb4a6137d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:35.482623 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccc0b3-2379-47c7-a6a5-1bcdb4a6137d admin admin] REQUEST_METHOD: `PUT` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:35.482911 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccc0b3-2379-47c7-a6a5-1bcdb4a6137d admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:35.483188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccc0b3-2379-47c7-a6a5-1bcdb4a6137d admin admin] PATH_INFO: `/v3/projects/573638f90aba47f18d1a91b5a2681677/users/c1bf37423f014b7b9b22f2aff0b364e3/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:35.499119 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccc0b3-2379-47c7-a6a5-1bcdb4a6137d admin admin] RBAC: Authorizing `identity:create_grant(project_id=573638f90aba47f18d1a91b5a2681677, user_id=c1bf37423f014b7b9b22f2aff0b364e3, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:35.501772 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccc0b3-2379-47c7-a6a5-1bcdb4a6137d admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:35.639798 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1430/2866] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:26:35 2022] PUT /identity/v3/projects/573638f90aba47f18d1a91b5a2681677/users/c1bf37423f014b7b9b22f2aff0b364e3/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:35.706405 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040ac10-84c3-4771-b772-945b44a4e08f None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:35.706746 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040ac10-84c3-4771-b772-945b44a4e08f None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:35.707017 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040ac10-84c3-4771-b772-945b44a4e08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:35.735091 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-b040ac10-84c3-4771-b772-945b44a4e08f None None] MFA Rules not processed for user `c1bf37423f014b7b9b22f2aff0b364e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:35.736347 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-b040ac10-84c3-4771-b772-945b44a4e08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:35.801392 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1437/2867] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:35.806099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccc0b3-2379-47c7-a6a5-1bcdb4a6137d admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:35.868504 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccc0b3-2379-47c7-a6a5-1bcdb4a6137d admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:35.873283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0b83e-fb9b-468e-9508-a5abffab2ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:35.873940 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0b83e-fb9b-468e-9508-a5abffab2ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:35.874226 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0b83e-fb9b-468e-9508-a5abffab2ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:35.874505 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0b83e-fb9b-468e-9508-a5abffab2ae3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:35.875221 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0b83e-fb9b-468e-9508-a5abffab2ae3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:35.877156 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0b83e-fb9b-468e-9508-a5abffab2ae3 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:35.965390 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1431/2868] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:26:35 2022] POST /identity/v3/projects => generated 375 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:26:35.969729 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040ac10-84c3-4771-b772-945b44a4e08f None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:35.981750 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040ac10-84c3-4771-b772-945b44a4e08f None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:36.039059 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad9b0c-0028-4b39-a3a8-fa817ab55f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:36.039732 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad9b0c-0028-4b39-a3a8-fa817ab55f2e admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:36.040019 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad9b0c-0028-4b39-a3a8-fa817ab55f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:36.040297 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad9b0c-0028-4b39-a3a8-fa817ab55f2e admin admin] PATH_INFO: `/v3/users` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:36.040994 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad9b0c-0028-4b39-a3a8-fa817ab55f2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:36.042962 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad9b0c-0028-4b39-a3a8-fa817ab55f2e admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:36.116839 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1438/2869] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:26:35 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:26:36.121030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0b83e-fb9b-468e-9508-a5abffab2ae3 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:36.131510 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0b83e-fb9b-468e-9508-a5abffab2ae3 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:36.136866 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ee723-10d0-476a-a3c9-bada125a287a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:36.137569 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ee723-10d0-476a-a3c9-bada125a287a admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:36.137874 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ee723-10d0-476a-a3c9-bada125a287a admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:36.138153 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ee723-10d0-476a-a3c9-bada125a287a admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:36.138904 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ee723-10d0-476a-a3c9-bada125a287a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:36.140521 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ee723-10d0-476a-a3c9-bada125a287a admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:36.151898 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1432/2870] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:26:36.156336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad9b0c-0028-4b39-a3a8-fa817ab55f2e admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:36.167076 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad9b0c-0028-4b39-a3a8-fa817ab55f2e admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:36.171671 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed6f41-112d-454a-9d0e-5d2db68b8a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:36.172583 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed6f41-112d-454a-9d0e-5d2db68b8a49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:36.172871 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed6f41-112d-454a-9d0e-5d2db68b8a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:36.173141 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed6f41-112d-454a-9d0e-5d2db68b8a49 admin admin] PATH_INFO: `/v3/projects/f6b729fa5d1b4be09f26e4f313085fe9/users/1be4e4a8947f4aaf8a5892e88c504f7b/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:36.185686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:36.188781 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed6f41-112d-454a-9d0e-5d2db68b8a49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6b729fa5d1b4be09f26e4f313085fe9, user_id=1be4e4a8947f4aaf8a5892e88c504f7b, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:36.191566 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed6f41-112d-454a-9d0e-5d2db68b8a49 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:36.275875 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1439/2871] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:26:36 2022] PUT /identity/v3/projects/f6b729fa5d1b4be09f26e4f313085fe9/users/1be4e4a8947f4aaf8a5892e88c504f7b/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:36.280144 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ee723-10d0-476a-a3c9-bada125a287a admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:36.290918 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ee723-10d0-476a-a3c9-bada125a287a admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:36.333025 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:36.349001 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088da6f-0311-4b55-a929-01c3f1096b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:36.349678 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088da6f-0311-4b55-a929-01c3f1096b51 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:36.349967 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088da6f-0311-4b55-a929-01c3f1096b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:36.350245 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088da6f-0311-4b55-a929-01c3f1096b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:36.351901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088da6f-0311-4b55-a929-01c3f1096b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:36.353534 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088da6f-0311-4b55-a929-01c3f1096b51 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:36.367914 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1433/2872] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:26:36.369215 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed6f41-112d-454a-9d0e-5d2db68b8a49 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:36.380402 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed6f41-112d-454a-9d0e-5d2db68b8a49 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:36.386396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893fbb0-152a-49c6-9236-580029325d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:36.387381 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893fbb0-152a-49c6-9236-580029325d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:36.387669 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893fbb0-152a-49c6-9236-580029325d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:36.387946 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893fbb0-152a-49c6-9236-580029325d5f admin admin] PATH_INFO: `/v3/projects/f6b729fa5d1b4be09f26e4f313085fe9/users/1be4e4a8947f4aaf8a5892e88c504f7b/roles/5891920310b54787bd1152d736d2a3f1` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:36.389965 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893fbb0-152a-49c6-9236-580029325d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6b729fa5d1b4be09f26e4f313085fe9, user_id=1be4e4a8947f4aaf8a5892e88c504f7b, role_id=5891920310b54787bd1152d736d2a3f1)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:36.393078 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893fbb0-152a-49c6-9236-580029325d5f admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:36.403900 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1440/2873] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:26:36 2022] PUT /identity/v3/projects/f6b729fa5d1b4be09f26e4f313085fe9/users/1be4e4a8947f4aaf8a5892e88c504f7b/roles/5891920310b54787bd1152d736d2a3f1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:36.414588 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d6cbf-6f41-4fea-a7c5-87e975452566 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:36.414913 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d6cbf-6f41-4fea-a7c5-87e975452566 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:36.415197 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d6cbf-6f41-4fea-a7c5-87e975452566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:36.442030 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-e39d6cbf-6f41-4fea-a7c5-87e975452566 None None] MFA Rules not processed for user `1be4e4a8947f4aaf8a5892e88c504f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:36.443196 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e39d6cbf-6f41-4fea-a7c5-87e975452566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:36.512300 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1434/2874] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:36.524063 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893fbb0-152a-49c6-9236-580029325d5f admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:36.525792 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-c893fbb0-152a-49c6-9236-580029325d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:36.583418 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893fbb0-152a-49c6-9236-580029325d5f admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:36.587124 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c3c08-7a38-4ea8-b376-716b8e09370d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:36.587767 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c3c08-7a38-4ea8-b376-716b8e09370d service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:36.588051 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c3c08-7a38-4ea8-b376-716b8e09370d service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:36.588330 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c3c08-7a38-4ea8-b376-716b8e09370d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:36.644521 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c3c08-7a38-4ea8-b376-716b8e09370d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:36.645963 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c3c08-7a38-4ea8-b376-716b8e09370d service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:36.655031 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1441/2875] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:26:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:26:38.613227 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d6cbf-6f41-4fea-a7c5-87e975452566 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:38.614834 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-e39d6cbf-6f41-4fea-a7c5-87e975452566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:38.627087 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d6cbf-6f41-4fea-a7c5-87e975452566 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:38.630738 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83710a5-8e18-496a-ae61-ff0d6bb56177 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:38.631380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83710a5-8e18-496a-ae61-ff0d6bb56177 service cinder] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:38.631662 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83710a5-8e18-496a-ae61-ff0d6bb56177 service cinder] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:38.631943 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83710a5-8e18-496a-ae61-ff0d6bb56177 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:38.687759 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83710a5-8e18-496a-ae61-ff0d6bb56177 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:38.689214 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83710a5-8e18-496a-ae61-ff0d6bb56177 service cinder] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:38.699050 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1435/2876] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:26:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:26:40.168669 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c3c08-7a38-4ea8-b376-716b8e09370d service cinder] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:40.177546 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c3c08-7a38-4ea8-b376-716b8e09370d service cinder] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:40.181829 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fcbfba-06ce-47cc-81ef-893c6343b1dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9218503bc8a647d68ffab6945bae456a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:40.182404 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fcbfba-06ce-47cc-81ef-893c6343b1dc service cinder] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:40.182723 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fcbfba-06ce-47cc-81ef-893c6343b1dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:40.183002 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fcbfba-06ce-47cc-81ef-893c6343b1dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:40.191938 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fcbfba-06ce-47cc-81ef-893c6343b1dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:40.193366 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fcbfba-06ce-47cc-81ef-893c6343b1dc service cinder] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:40.203349 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1442/2877] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:26:40 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:26:44.910201 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83710a5-8e18-496a-ae61-ff0d6bb56177 service cinder] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:44.974594 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83710a5-8e18-496a-ae61-ff0d6bb56177 service cinder] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:44.979984 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214b359-f2b6-4ba6-8592-23551a198276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:44.980872 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214b359-f2b6-4ba6-8592-23551a198276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:44.981147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214b359-f2b6-4ba6-8592-23551a198276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:44.981426 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214b359-f2b6-4ba6-8592-23551a198276 admin admin] PATH_INFO: `/v3/users/fcb82d098513497eb27c57fb1fec6c51` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:44.982713 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214b359-f2b6-4ba6-8592-23551a198276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcb82d098513497eb27c57fb1fec6c51)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:44.984610 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214b359-f2b6-4ba6-8592-23551a198276 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:44.989707 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=100 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:45.258965 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7214b359-f2b6-4ba6-8592-23551a198276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcb82d098513497eb27c57fb1fec6c51'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:45.265959 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7214b359-f2b6-4ba6-8592-23551a198276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcb82d098513497eb27c57fb1fec6c51'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:45.266708 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7214b359-f2b6-4ba6-8592-23551a198276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcb82d098513497eb27c57fb1fec6c51'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:45.280962 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7214b359-f2b6-4ba6-8592-23551a198276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcb82d098513497eb27c57fb1fec6c51'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:45.342694 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-7214b359-f2b6-4ba6-8592-23551a198276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcb82d098513497eb27c57fb1fec6c51'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:45.349787 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1436/2878] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:26:44 2022] DELETE /identity/v3/users/fcb82d098513497eb27c57fb1fec6c51 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:45.355177 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958974e-0ae0-4ea2-a512-7f4588c13dc8 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:45.355736 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958974e-0ae0-4ea2-a512-7f4588c13dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:45.355736 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958974e-0ae0-4ea2-a512-7f4588c13dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:45.374215 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-7958974e-0ae0-4ea2-a512-7f4588c13dc8 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:45.375505 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7958974e-0ae0-4ea2-a512-7f4588c13dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:45.435416 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1443/2879] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 15:26:45.446188 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214b359-f2b6-4ba6-8592-23551a198276 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:45.447640 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-7214b359-f2b6-4ba6-8592-23551a198276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:45.505166 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214b359-f2b6-4ba6-8592-23551a198276 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:45.509511 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640abc6-fe9c-4e2c-8774-4d9d3a025638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:45.510159 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640abc6-fe9c-4e2c-8774-4d9d3a025638 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:45.510447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640abc6-fe9c-4e2c-8774-4d9d3a025638 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:45.510755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640abc6-fe9c-4e2c-8774-4d9d3a025638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:45.519790 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a640abc6-fe9c-4e2c-8774-4d9d3a025638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:45.521235 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a640abc6-fe9c-4e2c-8774-4d9d3a025638 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:45.531432 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1437/2880] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:26:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:26:45.989692 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958974e-0ae0-4ea2-a512-7f4588c13dc8 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:45.991838 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-7958974e-0ae0-4ea2-a512-7f4588c13dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:46.006435 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958974e-0ae0-4ea2-a512-7f4588c13dc8 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:46.013185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce48870-49da-4ea1-932a-2bcf1af47d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:46.013185 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce48870-49da-4ea1-932a-2bcf1af47d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:46.013446 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce48870-49da-4ea1-932a-2bcf1af47d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:46.013598 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce48870-49da-4ea1-932a-2bcf1af47d41 admin admin] PATH_INFO: `/v3/projects/20432d0a46bc46129034faec8269572a` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:46.014866 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce48870-49da-4ea1-932a-2bcf1af47d41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20432d0a46bc46129034faec8269572a)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:46.016786 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce48870-49da-4ea1-932a-2bcf1af47d41 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:46.034219 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:46.168336 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0ce48870-49da-4ea1-932a-2bcf1af47d41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20432d0a46bc46129034faec8269572a', 'request_id': 'req-0ce48870-49da-4ea1-932a-2bcf1af47d41'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:46.174593 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0ce48870-49da-4ea1-932a-2bcf1af47d41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20432d0a46bc46129034faec8269572a', 'request_id': 'req-0ce48870-49da-4ea1-932a-2bcf1af47d41'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:46.181537 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0ce48870-49da-4ea1-932a-2bcf1af47d41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20432d0a46bc46129034faec8269572a', 'request_id': 'req-0ce48870-49da-4ea1-932a-2bcf1af47d41'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:46.255694 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0ce48870-49da-4ea1-932a-2bcf1af47d41 admin admin] The token cache is being invalidate because project 20432d0a46bc46129034faec8269572a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:26:46.255987 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0ce48870-49da-4ea1-932a-2bcf1af47d41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:46.256702 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-0ce48870-49da-4ea1-932a-2bcf1af47d41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:46.258304 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1444/2881] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:26:45 2022] DELETE /identity/v3/projects/20432d0a46bc46129034faec8269572a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:46.262447 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640abc6-fe9c-4e2c-8774-4d9d3a025638 service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:46.273938 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640abc6-fe9c-4e2c-8774-4d9d3a025638 service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:46.331067 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e665038-75ba-407f-bbbd-4bfbf03a5709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:46.331986 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e665038-75ba-407f-bbbd-4bfbf03a5709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:46.332283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e665038-75ba-407f-bbbd-4bfbf03a5709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:46.332563 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e665038-75ba-407f-bbbd-4bfbf03a5709 admin admin] PATH_INFO: `/v3/users/c1bf37423f014b7b9b22f2aff0b364e3` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:46.333885 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e665038-75ba-407f-bbbd-4bfbf03a5709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1bf37423f014b7b9b22f2aff0b364e3)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:46.336421 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e665038-75ba-407f-bbbd-4bfbf03a5709 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:46.455128 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:46.576147 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4e665038-75ba-407f-bbbd-4bfbf03a5709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1bf37423f014b7b9b22f2aff0b364e3'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:46.583233 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4e665038-75ba-407f-bbbd-4bfbf03a5709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1bf37423f014b7b9b22f2aff0b364e3'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:46.583919 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4e665038-75ba-407f-bbbd-4bfbf03a5709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1bf37423f014b7b9b22f2aff0b364e3'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:46.597213 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4e665038-75ba-407f-bbbd-4bfbf03a5709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1bf37423f014b7b9b22f2aff0b364e3'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:46.664755 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-4e665038-75ba-407f-bbbd-4bfbf03a5709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1bf37423f014b7b9b22f2aff0b364e3'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:46.671107 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1438/2882] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:26:46 2022] DELETE /identity/v3/users/c1bf37423f014b7b9b22f2aff0b364e3 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:47.138519 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce48870-49da-4ea1-932a-2bcf1af47d41 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:47.140114 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-0ce48870-49da-4ea1-932a-2bcf1af47d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:47.154653 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce48870-49da-4ea1-932a-2bcf1af47d41 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:47.214197 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f30c8d-d950-4eac-9ef5-0607c824d7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:47.215118 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f30c8d-d950-4eac-9ef5-0607c824d7aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:47.215413 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f30c8d-d950-4eac-9ef5-0607c824d7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:47.215695 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f30c8d-d950-4eac-9ef5-0607c824d7aa admin admin] PATH_INFO: `/v3/projects/573638f90aba47f18d1a91b5a2681677` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:47.216901 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f30c8d-d950-4eac-9ef5-0607c824d7aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=573638f90aba47f18d1a91b5a2681677)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:47.218880 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f30c8d-d950-4eac-9ef5-0607c824d7aa admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:47.352102 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-62f30c8d-d950-4eac-9ef5-0607c824d7aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '573638f90aba47f18d1a91b5a2681677', 'request_id': 'req-62f30c8d-d950-4eac-9ef5-0607c824d7aa'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:47.358686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-62f30c8d-d950-4eac-9ef5-0607c824d7aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '573638f90aba47f18d1a91b5a2681677', 'request_id': 'req-62f30c8d-d950-4eac-9ef5-0607c824d7aa'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:47.364444 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-62f30c8d-d950-4eac-9ef5-0607c824d7aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '573638f90aba47f18d1a91b5a2681677', 'request_id': 'req-62f30c8d-d950-4eac-9ef5-0607c824d7aa'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:47.442404 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-62f30c8d-d950-4eac-9ef5-0607c824d7aa admin admin] The token cache is being invalidate because project 573638f90aba47f18d1a91b5a2681677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:26:47.442707 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-62f30c8d-d950-4eac-9ef5-0607c824d7aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:47.443296 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-62f30c8d-d950-4eac-9ef5-0607c824d7aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:47.445664 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1445/2883] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:26:47 2022] DELETE /identity/v3/projects/573638f90aba47f18d1a91b5a2681677 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:47.449865 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e665038-75ba-407f-bbbd-4bfbf03a5709 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:47.451462 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-4e665038-75ba-407f-bbbd-4bfbf03a5709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:47.465921 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e665038-75ba-407f-bbbd-4bfbf03a5709 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:47.531113 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b383a0-889d-4e2c-b65c-a1782789287e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:47.532004 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b383a0-889d-4e2c-b65c-a1782789287e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:47.532286 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b383a0-889d-4e2c-b65c-a1782789287e admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:47.532714 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b383a0-889d-4e2c-b65c-a1782789287e admin admin] PATH_INFO: `/v3/users/1be4e4a8947f4aaf8a5892e88c504f7b` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:47.533963 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b383a0-889d-4e2c-b65c-a1782789287e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1be4e4a8947f4aaf8a5892e88c504f7b)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:47.536019 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b383a0-889d-4e2c-b65c-a1782789287e admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:47.742078 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a9b383a0-889d-4e2c-b65c-a1782789287e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1be4e4a8947f4aaf8a5892e88c504f7b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:47.749471 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a9b383a0-889d-4e2c-b65c-a1782789287e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1be4e4a8947f4aaf8a5892e88c504f7b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:47.750151 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a9b383a0-889d-4e2c-b65c-a1782789287e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1be4e4a8947f4aaf8a5892e88c504f7b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:47.762034 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a9b383a0-889d-4e2c-b65c-a1782789287e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1be4e4a8947f4aaf8a5892e88c504f7b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:47.853214 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a9b383a0-889d-4e2c-b65c-a1782789287e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1be4e4a8947f4aaf8a5892e88c504f7b'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:47.859792 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1439/2884] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:26:47 2022] DELETE /identity/v3/users/1be4e4a8947f4aaf8a5892e88c504f7b => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:48.326635 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f30c8d-d950-4eac-9ef5-0607c824d7aa admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:48.328256 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-62f30c8d-d950-4eac-9ef5-0607c824d7aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:48.343032 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f30c8d-d950-4eac-9ef5-0607c824d7aa admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:48.376941 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:48.404456 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786f84d-bf57-4dc7-9cd4-b211f65aee26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:48.405316 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786f84d-bf57-4dc7-9cd4-b211f65aee26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:48.405605 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786f84d-bf57-4dc7-9cd4-b211f65aee26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:48.405882 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786f84d-bf57-4dc7-9cd4-b211f65aee26 admin admin] PATH_INFO: `/v3/projects/f6b729fa5d1b4be09f26e4f313085fe9` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:48.407156 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786f84d-bf57-4dc7-9cd4-b211f65aee26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6b729fa5d1b4be09f26e4f313085fe9)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:48.409093 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786f84d-bf57-4dc7-9cd4-b211f65aee26 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:48.542895 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9786f84d-bf57-4dc7-9cd4-b211f65aee26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6b729fa5d1b4be09f26e4f313085fe9', 'request_id': 'req-9786f84d-bf57-4dc7-9cd4-b211f65aee26'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:48.549099 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9786f84d-bf57-4dc7-9cd4-b211f65aee26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6b729fa5d1b4be09f26e4f313085fe9', 'request_id': 'req-9786f84d-bf57-4dc7-9cd4-b211f65aee26'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:48.555354 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9786f84d-bf57-4dc7-9cd4-b211f65aee26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6b729fa5d1b4be09f26e4f313085fe9', 'request_id': 'req-9786f84d-bf57-4dc7-9cd4-b211f65aee26'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:48.632524 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9786f84d-bf57-4dc7-9cd4-b211f65aee26 admin admin] The token cache is being invalidate because project f6b729fa5d1b4be09f26e4f313085fe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:26:48.632775 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9786f84d-bf57-4dc7-9cd4-b211f65aee26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:48.633359 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-9786f84d-bf57-4dc7-9cd4-b211f65aee26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:26:48.634956 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1446/2885] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:26:48 2022] DELETE /identity/v3/projects/f6b729fa5d1b4be09f26e4f313085fe9 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:26:48.640943 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce11e9-8bce-41c0-9542-5f9a39217551 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:48.641494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce11e9-8bce-41c0-9542-5f9a39217551 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:48.641494 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce11e9-8bce-41c0-9542-5f9a39217551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:48.660168 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-97ce11e9-8bce-41c0-9542-5f9a39217551 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:48.661379 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-97ce11e9-8bce-41c0-9542-5f9a39217551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:48.680992 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:48.722774 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1440/2886] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:48.732327 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb31e2-2429-4d2c-8e04-c27c646a9472 None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:48.732630 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb31e2-2429-4d2c-8e04-c27c646a9472 None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:48.732917 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb31e2-2429-4d2c-8e04-c27c646a9472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:48.751541 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-88fb31e2-2429-4d2c-8e04-c27c646a9472 None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:48.752759 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-88fb31e2-2429-4d2c-8e04-c27c646a9472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:48.764447 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1447/2887] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:48.768860 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce11e9-8bce-41c0-9542-5f9a39217551 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:48.779077 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce11e9-8bce-41c0-9542-5f9a39217551 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:48.783667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f72e51-97aa-4b81-a6e2-cda029c145d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:48.784317 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f72e51-97aa-4b81-a6e2-cda029c145d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:48.784604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f72e51-97aa-4b81-a6e2-cda029c145d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:48.784880 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f72e51-97aa-4b81-a6e2-cda029c145d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:48.785603 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f72e51-97aa-4b81-a6e2-cda029c145d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80303) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 15:26:48.785905 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f72e51-97aa-4b81-a6e2-cda029c145d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:48.787630 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f72e51-97aa-4b81-a6e2-cda029c145d4 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:48.799240 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1441/2888] 10.222.0.83 () {58 vars in 1152 bytes} [Mon Jul 18 15:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 15:26:48.803755 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb31e2-2429-4d2c-8e04-c27c646a9472 None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:48.813144 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb31e2-2429-4d2c-8e04-c27c646a9472 None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:48.817856 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d97cb1-823b-4e1d-86da-1bdef09d6283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:48.818512 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d97cb1-823b-4e1d-86da-1bdef09d6283 admin admin] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:48.819368 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d97cb1-823b-4e1d-86da-1bdef09d6283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:48.819609 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d97cb1-823b-4e1d-86da-1bdef09d6283 admin admin] PATH_INFO: `/v3/projects` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:48.820396 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d97cb1-823b-4e1d-86da-1bdef09d6283 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:48.822326 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d97cb1-823b-4e1d-86da-1bdef09d6283 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:48.933252 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1448/2889] 10.222.0.83 () {60 vars in 1152 bytes} [Mon Jul 18 15:26:48 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:26:48.937604 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f72e51-97aa-4b81-a6e2-cda029c145d4 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:48.948834 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f72e51-97aa-4b81-a6e2-cda029c145d4 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:49.008732 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d206b-128e-4c0f-bb17-7b078c4ac937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:49.009393 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d206b-128e-4c0f-bb17-7b078c4ac937 admin admin] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:49.009679 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d206b-128e-4c0f-bb17-7b078c4ac937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:49.009956 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d206b-128e-4c0f-bb17-7b078c4ac937 admin admin] PATH_INFO: `/v3/users` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:49.010765 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d206b-128e-4c0f-bb17-7b078c4ac937 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:49.016580 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d206b-128e-4c0f-bb17-7b078c4ac937 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:49.119337 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1442/2890] 10.222.0.83 () {60 vars in 1143 bytes} [Mon Jul 18 15:26:48 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 15:26:49.123591 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d97cb1-823b-4e1d-86da-1bdef09d6283 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:49.134307 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d97cb1-823b-4e1d-86da-1bdef09d6283 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:49.139022 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2402bca-3b5e-46dc-ad39-6d72f64ca3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:49.139680 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2402bca-3b5e-46dc-ad39-6d72f64ca3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:49.139967 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2402bca-3b5e-46dc-ad39-6d72f64ca3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:49.140271 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2402bca-3b5e-46dc-ad39-6d72f64ca3a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:49.140993 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2402bca-3b5e-46dc-ad39-6d72f64ca3a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:49.142626 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2402bca-3b5e-46dc-ad39-6d72f64ca3a7 admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:49.153788 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1449/2891] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:26:49.158015 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d206b-128e-4c0f-bb17-7b078c4ac937 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:49.168955 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d206b-128e-4c0f-bb17-7b078c4ac937 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:49.173645 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab484970-ea31-4ea9-bf55-784becefd3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:49.174307 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab484970-ea31-4ea9-bf55-784becefd3ea admin admin] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:49.174620 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab484970-ea31-4ea9-bf55-784becefd3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:49.174903 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab484970-ea31-4ea9-bf55-784becefd3ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:49.175627 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab484970-ea31-4ea9-bf55-784becefd3ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:49.177253 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab484970-ea31-4ea9-bf55-784becefd3ea admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:49.187368 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1443/2892] 10.222.0.83 () {58 vars in 1121 bytes} [Mon Jul 18 15:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 15:26:49.191553 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2402bca-3b5e-46dc-ad39-6d72f64ca3a7 admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:49.202024 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2402bca-3b5e-46dc-ad39-6d72f64ca3a7 admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:49.206636 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ed88f-dbc1-463d-a6a7-fbf604cbc34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:49.207511 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ed88f-dbc1-463d-a6a7-fbf604cbc34c admin admin] REQUEST_METHOD: `PUT` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:49.207795 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ed88f-dbc1-463d-a6a7-fbf604cbc34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:49.208073 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ed88f-dbc1-463d-a6a7-fbf604cbc34c admin admin] PATH_INFO: `/v3/projects/f528faafb60e41abaaee59ff84317deb/users/6d55b073c4954b39bef60e22acf641d4/roles/3ab0ee91b3284a65a0dd5696a6250883` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:49.223904 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ed88f-dbc1-463d-a6a7-fbf604cbc34c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f528faafb60e41abaaee59ff84317deb, user_id=6d55b073c4954b39bef60e22acf641d4, role_id=3ab0ee91b3284a65a0dd5696a6250883)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:49.227549 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ed88f-dbc1-463d-a6a7-fbf604cbc34c admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:49.328642 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1450/2893] 10.222.0.83 () {60 vars in 1482 bytes} [Mon Jul 18 15:26:49 2022] PUT /identity/v3/projects/f528faafb60e41abaaee59ff84317deb/users/6d55b073c4954b39bef60e22acf641d4/roles/3ab0ee91b3284a65a0dd5696a6250883 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 15:26:49.340818 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe754da8-06e9-495f-be49-b04a46e3ac54 None None] REQUEST_METHOD: `POST` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:49.341283 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe754da8-06e9-495f-be49-b04a46e3ac54 None None] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:49.341734 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe754da8-06e9-495f-be49-b04a46e3ac54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:49.370255 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.auth.core [None req-fe754da8-06e9-495f-be49-b04a46e3ac54 None None] MFA Rules not processed for user `6d55b073c4954b39bef60e22acf641d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80303) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:26:49.371598 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-fe754da8-06e9-495f-be49-b04a46e3ac54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:49.441568 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1444/2894] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:26:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:26:49.454957 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ed88f-dbc1-463d-a6a7-fbf604cbc34c admin admin] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:26:49.456683 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a15ed88f-dbc1-463d-a6a7-fbf604cbc34c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:26:49.526973 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:49.540797 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ed88f-dbc1-463d-a6a7-fbf604cbc34c admin admin] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:26:49.544686 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4417339-f4d0-4456-b44e-13b2b50fd138 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8dcb0a0f2c4a2d955544324213739c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:26:49.545365 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4417339-f4d0-4456-b44e-13b2b50fd138 service glance] REQUEST_METHOD: `GET` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:26:49.545655 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4417339-f4d0-4456-b44e-13b2b50fd138 service glance] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:26:49.545923 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4417339-f4d0-4456-b44e-13b2b50fd138 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:26:49.555222 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4417339-f4d0-4456-b44e-13b2b50fd138 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:26:49.556692 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4417339-f4d0-4456-b44e-13b2b50fd138 service glance] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:26:49.566119 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1451/2895] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:26:49 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:26:59.438667 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:26:59.644223 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=8 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:27:18.133602 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe754da8-06e9-495f-be49-b04a46e3ac54 None None] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:27:18.196794 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe754da8-06e9-495f-be49-b04a46e3ac54 None None] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:27:18.201296 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c5955-69b3-4329-bb88-3d5ec5146439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:27:18.202171 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c5955-69b3-4329-bb88-3d5ec5146439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:27:18.202453 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c5955-69b3-4329-bb88-3d5ec5146439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:27:18.202767 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c5955-69b3-4329-bb88-3d5ec5146439 admin admin] PATH_INFO: `/v3/users/6d55b073c4954b39bef60e22acf641d4` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:27:18.203976 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c5955-69b3-4329-bb88-3d5ec5146439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d55b073c4954b39bef60e22acf641d4)` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:27:18.205979 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c5955-69b3-4329-bb88-3d5ec5146439 admin admin] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:27:18.432405 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a55c5955-69b3-4329-bb88-3d5ec5146439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d55b073c4954b39bef60e22acf641d4'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:27:18.439271 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a55c5955-69b3-4329-bb88-3d5ec5146439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d55b073c4954b39bef60e22acf641d4'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:27:18.439901 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a55c5955-69b3-4329-bb88-3d5ec5146439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d55b073c4954b39bef60e22acf641d4'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:27:18.451877 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a55c5955-69b3-4329-bb88-3d5ec5146439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d55b073c4954b39bef60e22acf641d4'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:27:18.529194 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.notifications [None req-a55c5955-69b3-4329-bb88-3d5ec5146439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d55b073c4954b39bef60e22acf641d4'} {{(pid=80303) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:27:18.535718 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1445/2896] 10.222.0.83 () {58 vars in 1223 bytes} [Mon Jul 18 15:27:18 2022] DELETE /identity/v3/users/6d55b073c4954b39bef60e22acf641d4 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:27:18.541204 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186e38a-e127-41bb-85f9-04dbd438b02e None None] REQUEST_METHOD: `POST` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:27:18.541730 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186e38a-e127-41bb-85f9-04dbd438b02e None None] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:27:18.541851 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186e38a-e127-41bb-85f9-04dbd438b02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:27:18.559478 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.auth.core [None req-a186e38a-e127-41bb-85f9-04dbd438b02e None None] MFA Rules not processed for user `baf04811d48d4c3fa638546a709d7179`. Rule list: `[]` (Enabled: `True`). {{(pid=80304) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 15:27:18.560716 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a186e38a-e127-41bb-85f9-04dbd438b02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:27:18.621285 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1452/2897] 10.222.0.83 () {58 vars in 957 bytes} [Mon Jul 18 15:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 15:27:18.631426 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c5955-69b3-4329-bb88-3d5ec5146439 admin admin] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:27:18.632858 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-a55c5955-69b3-4329-bb88-3d5ec5146439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:27:18.697586 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c5955-69b3-4329-bb88-3d5ec5146439 admin admin] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:27:18.701166 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1835f99-905d-4a1f-a82c-384a166f3689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd86979e0f0d4a4e9921ee5704901f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:27:18.701813 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1835f99-905d-4a1f-a82c-384a166f3689 service neutron] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:27:18.702099 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1835f99-905d-4a1f-a82c-384a166f3689 service neutron] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:27:18.702380 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1835f99-905d-4a1f-a82c-384a166f3689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:27:18.712153 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1835f99-905d-4a1f-a82c-384a166f3689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:27:18.713602 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1835f99-905d-4a1f-a82c-384a166f3689 service neutron] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:27:18.723418 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1446/2898] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:27:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:27:19.183100 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186e38a-e127-41bb-85f9-04dbd438b02e None None] Authenticating user token {{(pid=80304) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:27:19.184714 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.fernet_utils [None req-a186e38a-e127-41bb-85f9-04dbd438b02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80304) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:27:19.197412 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186e38a-e127-41bb-85f9-04dbd438b02e None None] Validating token access rules against request {{(pid=80304) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:27:19.202197 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99067389-4371-48c9-b440-d9dff4a4399e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'baf04811d48d4c3fa638546a709d7179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8015bf50f271424ca62c239a2df319f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80304) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:27:19.203110 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99067389-4371-48c9-b440-d9dff4a4399e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:27:19.203399 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99067389-4371-48c9-b440-d9dff4a4399e admin admin] SCRIPT_NAME: `/identity` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:27:19.203681 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99067389-4371-48c9-b440-d9dff4a4399e admin admin] PATH_INFO: `/v3/projects/f528faafb60e41abaaee59ff84317deb` {{(pid=80304) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:27:19.204940 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99067389-4371-48c9-b440-d9dff4a4399e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f528faafb60e41abaaee59ff84317deb)` {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:27:19.206908 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99067389-4371-48c9-b440-d9dff4a4399e admin admin] RBAC: Authorization granted {{(pid=80304) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:27:19.311376 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-99067389-4371-48c9-b440-d9dff4a4399e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f528faafb60e41abaaee59ff84317deb', 'request_id': 'req-99067389-4371-48c9-b440-d9dff4a4399e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:27:19.318521 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-99067389-4371-48c9-b440-d9dff4a4399e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f528faafb60e41abaaee59ff84317deb', 'request_id': 'req-99067389-4371-48c9-b440-d9dff4a4399e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:27:19.324096 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-99067389-4371-48c9-b440-d9dff4a4399e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f528faafb60e41abaaee59ff84317deb', 'request_id': 'req-99067389-4371-48c9-b440-d9dff4a4399e'} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:27:19.394830 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-99067389-4371-48c9-b440-d9dff4a4399e admin admin] The token cache is being invalidate because project f528faafb60e41abaaee59ff84317deb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80304) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 15:27:19.397263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-99067389-4371-48c9-b440-d9dff4a4399e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:27:19.397263 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG keystone.notifications [None req-99067389-4371-48c9-b440-d9dff4a4399e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80304) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 15:27:19.397729 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: [pid: 80304|app: 0|req: 1453/2899] 10.222.0.83 () {58 vars in 1232 bytes} [Mon Jul 18 15:27:19 2022] DELETE /identity/v3/projects/f528faafb60e41abaaee59ff84317deb => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 15:27:28.718895 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:27:29.391333 dv-cinder-836973-13-iscsi devstack@keystone.service[80304]: DEBUG dbcounter [-] [80304] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80304) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 15:31:19.979248 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1835f99-905d-4a1f-a82c-384a166f3689 service neutron] Authenticating user token {{(pid=80303) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 15:31:19.980675 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-d1835f99-905d-4a1f-a82c-384a166f3689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:31:20.123835 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1835f99-905d-4a1f-a82c-384a166f3689 service neutron] Validating token access rules against request {{(pid=80303) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 15:31:20.185455 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe1548-f51c-4a2a-bfcd-3561b01efa8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76cc0ae48b5a49c6a7a1c11a08c389e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f777cd6a0c744e594f8e29ab58cf8bd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80303) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 15:31:20.186026 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe1548-f51c-4a2a-bfcd-3561b01efa8f service placement] REQUEST_METHOD: `GET` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 15:31:20.186309 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe1548-f51c-4a2a-bfcd-3561b01efa8f service placement] SCRIPT_NAME: `/identity` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 15:31:20.186615 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe1548-f51c-4a2a-bfcd-3561b01efa8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80303) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 15:31:20.188423 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.fernet_utils [None req-cffe1548-f51c-4a2a-bfcd-3561b01efa8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80303) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 15:31:20.200082 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe1548-f51c-4a2a-bfcd-3561b01efa8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 15:31:20.201663 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe1548-f51c-4a2a-bfcd-3561b01efa8f service placement] RBAC: Authorization granted {{(pid=80303) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 15:31:20.211937 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: [pid: 80303|app: 0|req: 1447/2900] 10.222.0.83 () {60 vars in 1365 bytes} [Mon Jul 18 15:31:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 15:31:30.207350 dv-cinder-836973-13-iscsi devstack@keystone.service[80303]: DEBUG dbcounter [-] [80303] Writing DB stats keystone:SELECT=89 {{(pid=80303) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}